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(57) JPEG encoded data are entropy-decoded to 
quantized DOT coefficients which are sent to a predic- 
tion tap extraction circuit (41) and to a class tap extrac- 
tion circuit (42). The prediction tap extraction circuit (41 ) 
and the class tap extraction circuit (42) extract what Is 
needed from the quantized DCT coefficients to form pre- 
diction taps and class taps. A classification circuit (43) 
effects classification based on the class tape. A coeffi- 
cient table storage unit (44) sends tap coefficients cor- 
responding to the classes resulting from the classifica- 
tion to a sum of products circuit (45) , which sum of prod- 
ucts circuit (45) then effects linear predictive calcula- 
tions, using the tap coefficients and the class taps, to 
generate decoded picture data. 
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Description 

Technical Field 

5 [0001 ] This Invention relates to a data processing method and apparatus and to a recording medium. More particu- 
larly, it relates to a data processing method and apparatus and a recording medium which may be used with advantage 
for decoding e.g.. Irreversibly compressed pictures. 

Background Art 

10 

[0002] Digital picture data, for example, are exorbitant In data volume, so that, if the data Is to be recorded or trans- 
mitted directly, a recording medium or a transmission medium of an extremely large capacity Is required. Thus, In 
general, picture data are compression-coded for diminishing the data volume before recording or transmission. 
[0003] Among the systems for compression -coding a picture, there are a JPEG (Joint Photographic Experts Group) 
system, as a compression encoding system for a still picture, and an MPEG (Moving Picture Experts Group) system, 
as a compression encoding system for a moving picture. 

[0004] For example, the encoding and decoding of picture data In accordance with the JPEG system are performed 
as shown In Fig. 1. 

[0005] Fig. 1 A shows an illustrative structure of a conventional JPEG encoding device. 

20 [0006] Picture data to be encoded are Input to a blocking circuit 1 which then divides the Input picture data Into plural 
blocks each composed of 8x8 = 64 pixels. The blocks obtained In the blocking circuit 1 are routed to a discrete cosine 
transform (DCT) circuit 2. This DCT circuit 2 applies DCT processing to each block from the blocking circuit 1 to trans- 
form the block Into a sum total of 64 DCT coefficients comprised of a sole DC (direct current) component and 63 
frequency (AC(altematIng current)) components. The block-based 64 DCT coefficients are routed from the DCT circuit 

25 2 to a quantization circuit 3. 

[0007] The quantization circuit 3 quantizes DCT coefficients In accordance with a preset quantization table to route 
the results of quantization, referred to below as quantized DCT coefficients, along with the quantization table used for 
quantization to an entropy encoding circuit 4. 

[0008] Fig. 1 B shows an Instance of a quantization table used in the quantization circuit 3. In the quantization table, 
30 the quantization stepsare set In general In order to take characteristics of the human visual system Into account so 
that low frequency DCT coefficients, which are more crucial, are quantized finely, whereas high frequency DCT coef- 
ficients, which are less crucial, are quantized only coarsely. This suppresses deterioration of the picture quality of a 
picture to assure efficient compression. 

[0009] The entropy encoding circuit 4 applies entropy encoding processing, such as Huffman encoding, to the quan- 
35 tlzed DCT coefficients, from the quantization circuit 3 and appends the quantization table from the quantization circuit 
3 to output the resulting encoded data as JPEG encoding data. 

[0010] Fig. 1 C shows the structure of an instance of a conventional JPEG decoding device 1 0 used for decoding the 
encoded data output by a JPEG encoding device 5 of Flg.1 A. 

[001 1 ] The encoded data are input to an entropy decoding circuit 1 1 , which then separates the encoded data Into 
40 entropy encoded quantized DCT coefficients and the quantization table. The entropy decoding circuit 11 entropy de- 
codes the quantized entropy encoded DCT coefficients to route the resulting quantized DCT coefficients along with 
the quantization table to a dequanuzatlon circuit 12. The dequanttzatlon circuit 12 dequantlzes the quantized DCT 
coefficients from the entropy decoding circuit 11 , in accordance with the quantization table from the entropy decoding 
circuit 1 1 , to send the resulting DCT coefficients to an Inverse DCT circuit 13. The inverse DCT circuit 1 3 applies Inverse 
*3 DCT processing to the DCT coefficients from the dequantlzatlon circuit 12 to send the resulting 8X8 pixel decoded 
block to a deblocking circuit 13. The deblocking circuit 13 deblocks the decoded blocks from the Inverse DCT circuit 
1 3 to derive an output decoded picture. 

[0012] With the JPEG encoding device 5, shown In Flg.1 A, the data volume of the encoded data can be diminished 
by increasing the quantization step of the quantization table used in quantizing the block, thereby realizing high com- 
so presslon. 

[0013] However, rf the quantization step Is Increased, the so-called quantization error is also Increased to deteriorate 
the picture quality of the decoded picture obtained In the JPEG decoding device 10 of Rg.1C. That Is, blurring, block 
distortion or the mosquito noise etc are produced appreciably In the decoded picture. 

[0014] Thus, If the data volume of the encoded data is to be diminished but nevertheless the picture quality of the 
55 decoded is not to be deteriorated, or if the data volume ofthe encoded data is maintained as the picture quality ofthe 
decoded picture is to be improved, some processing or other for improving the picture quality needs to be carried out 
subsequent to JPEG decoding. 

[0015] However, If processing for Improving the picture quality is to be performed subsequent to JPEG decoding, 
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the processing Is complicated to prolong the time until the decoded picture Is ultimately produced. 
Disclosure of the Invention 

5 [00161 In view of the above-described status of the art, the present Invention contemplates to enable a decoded 
picture of high picture quality to be produced efficiently from e.g., a JPEG encoded picture. 

[0017J According to the present Invention, tap coefficients as found on learning are acquired, and the so acquired 
tap coefficients are used along with the converted data to execute preset calculations to decode the converted data 
to original data. 

10 [00181 That Is, the present Invention provides a data processing apparatus Including acquisition means for acquiring 
tap coefficients as found by learning, and decoding means for decoding the converted data to original data by preset 
predictive calculations employing the tap coefficients and the converted data. 

[001 91 The present Invention also provides a data processing method including the steps of acquiring tap coefficients 
as found by learning, and decoding the converted data to original data by preset predictive calculations employing the 
13 tap coefficients and the converted data. 

[0020] The present invention also provides a recording medium having recorded thereon a program Including the 
steps of acquiring tap coefficients as found by learning, and decoding the converted data to original data by preset 
predictive calculations employing the tap coefficients and the converted data. 

[0021] According to the present invention, pupil data, as pupil, are generated on at least orthogonal or frequency 
20 transforming teacher data as teacher, and tap coefficients are found by effecting learning so that the prediction error 
of prediction value of the teacher data obtained on predictive calculations using the tap coefficients and the pupil data 
will be statistically minimum. 

[0022] That Is, the present Invention also provides a data processing apparatus Including generating means for 
generating pupil data, as pupil, by at least orthogonal or frequency transforming teacher data, as teacher, and learning 

25 means for effecting learning so that a prediction error of prediction value of the teacher data obtained on predictive 
calculations using the tap coefficients and the pupil data will be statistically minimum, to find the tap coefficients. 
[0023] The present Invention also provides a data processing method Including the steps of generating pupil data, 
as pupil, by at least orthogonal or frequency transforming teacher data, as teacher, and effecting learning so that a 
prediction error of the prediction value of the teacher data obtained on predictive calculations using the tap coefficients 

oo and the pupil data will be statistically minimum to find the tap coefficients. 

[0024] The present invention also provides a recording medium having recorded thereon a program including the 
steps of generating pupil data, as pupil, by at least orthogonal or frequency transforming teacher data, as teacher, and 
effecting learning so that a prediction error of the prediction value of the teacher data obtained on predictive calculations , 
using the tap coefficients and the pupil data will be statistically minimum to find the tap coefficients. 

35 [00251 According to the present Invention, tap coefficients as found on learning are acquired, and preset predictive 
calculations are executed using the tap coefficients and converted data to decode the converted data to original data, 
which original data is processed in a preset fashion to produce processed data. 

[0026] That Is. the present Invention also provides a data processing apparatus Including acquisition means for 
acquiring tap coefficients astound by learning, and calculating means for executing preset predictive calculations using 
40 the tap coefficients and the converted data to decode the converted data to original data and to produce processed 
data which is the original data subjected to preset processing. 

[0027] The present Invention also provides a data processing method Including the steps of acquiring tap coefficients 
as found by learning, and executing preset predictive calculations using the tap coefficients and the converted data to 
decode the converted data to original data and to produce processed data which Is the original data subjected to preset 
45 processing. 

[0028] The present Invention also provides a recording medium having recorded thereon a program Including the 
steps of acquiring tap coefficients as found by learning, and executing preset predictive calculations using the tap 
coefficients and the converted data to decode the converted data to original data and to produce processed data which 
is the original data subjected to preset processing. 

so [0029] According to the present invention , teacher data as teacher are processed In a preset fashion and the resulting 
quasl-teacher data are at least orthogonal or frequency transformed to generate pupil data as pupil. Tap coefficients 
are then generated on effecting learning so that a a prediction error of the predicted value of the teacher- data obtained 
on predictive calculations employing the tap coefficients and the pupil data will be statistically minimum. 
[0030J That Is, the present Invention also provides a data processing apparatus Including quasl-teacher data gen- 

55 erating means for processing teacher data as teacher, based on the preset processing, to produce quasi-teacher data, 
pupil data generating means for generating pupil data as pupiis on at least orthogonal or frequency transforming the 
quasl-teacher data, and learning means for effecting learning for statistically minimizing a prediction error of the pre- 
dicted value of the teacher data obtained on predictive calculations employing the tap coefficients and the pupil data 



3 



EP 1 282 236 A1 



to And the tap coefficients. 

[0031 ] The present Invention also provides a data processing method Including the steps of processing teacher data 
as teacher, based on the preset processing, to produce quasi -teacher data, generating pupil data as pupils on at least 
orthogonal or frequency transforming the quasl-teacher data and effecting learning for statistically minimizing the pre- 
s diction error of the predicted value of the teacher data obtained on predictive calculations employing the tap coefficients 
and the pupil data to find the tap coefficients. 

[0032] The present invention also provides a recording medium having recorded thereon a program Including the 
steps of processing teacher data as teacher, based on the preset processing, to produce quasl-teacher data, generating 
pupil data as pupils on at least orthogonal or frequency transforming the quasl-teacher data and effecting learning for 
10 statistically minimizing the prediction error of the predicted value of the teacher data obtained on predictive calculations 
employing the tap coefficients and the pupil data to find the tap coefficients. 

[0033] According to the present Invention, data of Interest among original data Is assigned to one of a plural number 
of classes, based on subsidiary information, and tap coefficients corresponding to the class of the data of interest, 
among class-based tap coefficients, as found on learning, are acquired. Preset predictive calculations are then carried 
« out, using the converted data and the tap coefficients corresponding to the dass of the data of Interest, to decode the 
converted data to original data. 

[0034] That Is, the present invention also provides a data processing apparatus including classification means for 
assigning data of Interest In the original data to one of a plurality of classes, based on the subsidiary Information, by 
way of classification, acquisition means for acquiring tap coefficients associated with the class of the data of interest, 
20 in class-based tap coefficients as found on learning and decoding means for executing preset predictive calculations, 
using the converted data and the tap coefficients associated with the class of the data of Interest, to decode the con- 
verted data into original data. 

[0035] The present invention also provides a data processing method including the steps of assigning data of Interest 
In the original data to one of a plurality of classes, based on the subsidiary Information, byway of classification, acquiring 
25 tap coefficients associated with the class of the data of interest, in class-based tap coefficients as found on learning 
and executing preset predictive calculations, using the converted data and the tap coefficients associated with the 
class of the data of Interest, to decode the converted data Into original data. 

[0036] The present invention also provides a recording medium having recorded thereon a program Including the 
steps of assigning data of interest in the original data to one of a plurality of classes, based on the subsidiary information, 

30 by way of classification, acquiring tap coefficients associated with the class of the data of Interest, In class-based tap 
coefficients as found on learning, and executing preset predictive calculations, using the converted data and the tap 
coefficients associated with the dass of the data of Interest, to decode the converted data Into original data. 
[0037] According to the present Invention, pupil data, as pupil, are generated on at least orthogonal or frequency 
transforming teacher data as teacher. Uslngthe preset subsidiary Information, used In generating the pupil data, teacher 

3ff data of interest, among the teacher data, is assigned to one of a plural n umber of classes. Class-based tap coeff Idents 
are then found by effecting learning so that a prediction error of the prediction value of the teacher data obtained on 
predictive calculations using the tap coefficients corresponding to the class of the teacher data of Interest and pupil 
data will be statistically minimum. 

[0033] That is, the present invention also provides a data processing apparatus Induding generating means for 
40 generating pupil data, as pupil, at least on orthogonal or frequency transforming teacher data as teacher, classification 
means for assigning teacher data of Interest in the teacher data to one of a plurality of dasses, based on the preset 
subsidiary Information used In generating the pupil data In the generating means, and learning means for effecting 
learning so that a prediction error of the prediction value of the teacher data obtained on predictive calculations using 
the tap coefficients corresponding to the class of the teacher data of interest and pupil data win be statistically minimum 
45 to find dass-based tap coefficients. 

[0039] The present Invention also provides a data processing method induding the steps of generating pupil data, 
as pupil, at least on orthogonal or frequency transforming teacher data as teacher, assigning teacher data of Interest 
In the teacher data to one of a plurality of dasses based on the preset subsidiary Information used in generating the 
pupil data in the generating means and effecting learning so that a prediction error of the prediction value of the teacher 
50 data obtained on predictive calculations using the tap coefficients corresponding to the dass of the teacher data of 
Interest and pupil data will be statistically minimum, to find dass-based tap coeffldents. 

[0040] The present Invention also provides a recording medium having recorded thereon a program Including the 
steps of generating pupO data, as pupil, at least on orthogonal or frequency transforming teacher data as teacher, 
assigning teacher data of Interest in the teacher data to one ot a plurality of dasses based on the preset subsidiary 
55 information used in generating the pupil data in the generating means and effecting learning so that a prediction error 
of the prediction value of the teacher data obtained on predictive calculations using the tap coeffldents corresponding 
to the dass of the teacher data of Interest and pupil data will be statistically minimum, to And class-based tap coeffldents. 
[0041 ] According to the present Invention, converted data used In assigning processed data of Interest to one of a 
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plural number of classes is extracted, at least from a block other than a block corresponding to processed data of 
Interest, for use as class tap. Based on the class tap, thus found, the class of the processed data of Interest is found 
by way of classification. Using the tap coefficients of the class of the processed data of interest, preset calculations 
are carried out to find a prediction value of the processed data of Interest. 

5 [0042] That is, the present invention also provides a processing apparatus Including acquisition means for acquiring 
tap coefficients as found on learning, class tap extraction means for extracting the converted data, used for assigning 
processed data of interest, to one of a plurality of classes, from at least a block other than the block associated with 
the processed data of interest, by way of classification, and for outputting thB extracted data as class tap, classification 
means for finding a class ofthe processed data of interest based on the class tap, and calculating means for executing 

10 preset predictive calculations, using the tap coefficients ofthe class ofthe processed data of Interest, and the converted 
data, to find a prediction value of the processed data of Interest. 

[0043] The present invention also provides a data processing method including the steps of acquiring tap coefficients 
as found on learning, extracting the converted data, used for assigning processed data of interest, to one of a plurality 
of classes, from at least a block other than the block associated with the processed data of interest, by way of classi- 
fy fbation, and for outputting the extracted data as class tap, finding a class of the processed data of interest based on 
the class tap, and executing preset predictive calculations, using the tap coefficients of the class of the processed data 
of interest, to find a prediction value of the processed data of Interest 

[0044] The present Invention also provides a recording medium Including the steps of acquiring tap coefficients as 
found on learning, extracting the converted data, used for assigning processed data of interest, to one of a plurality ol 
20 classes , from at least a block other than the block associated with the processed data of interest, by way of classification , 
and for outputting the extracted data as class tap, and finding a class of the processed data of interest based on the 
class tap, and executing preset predictive calculations, using the tap coefficients of the class of the processed data ol 
Interest, to find a prediction value of the processed data of Interest. 

[0045] According to the present Invention, pupil data used for assigning teacher data of Interest to one of a plural 
25 number of classes, are extracted at least from a block other than a block corresponding to teacher data of interest, for 
use as class taps. Based on these class taps, the class of the reacher data of Interest Is found by way of classification. 
Predictive calculations are carried out based on class-based tap coefficients and the pupil data and learning is then 
carried out so that a prediction error of the prediction value of the teacher data obtained on predictive calculations 
using the class-based tap coefficients and pupil data will be statistically minimum, to find tap coefficients from class to 
30 class. 

[0046] That is, the present invention also provides a data processing apparatus including generating means for 
generating the block-based converted data as pupil data as pupil In learning, class tap extraction means for extracting 
pupil data used for assigning teacher data of interest to one of a plurality of classes, at least from a block other than 
a block corresponding to the teacher data of interest, with the processed data as teacher data as teacher in learning, 
35 and for outputting the extracted data as class tap, classification means for finding a class of the teacher data of interest, 
based on the class tap, and learning means for effecting learning so that a prediction error of the prediction value of 
the teacher data obtained on predictive calculations using the class-based tap coefficients and pupil data will be sta- 
tistically minimum to find the tap coefficients Irom class to class. 

[0047] The present Invention also provides a data processing method including the steps of generating the block- 

40 based converted data as pupil data as pupil In learning, extracting pupil data used for assigning teacher data of interest 
to one of a plurality of classes, at least from a block other than a block corresponding to the teacher data of Interest, 
with the processed data as teacher data as teacher in learning, and for outputting the extracted data as class tap, 
finding a class of the teacher data of interest, based on the class tap, and effecting learning so that a prediction error 
of the prediction value of the teacher data obtained on predictive calculations using the class-based tap coefficients 

45 and pupil data will be statistically minimum to find the tap coefficients from class to class. 

[0048] The present invention also provides a recording medium having recorded thereon a program including the 
steps of generating the block-based converted data as pupil data as pupil In learning, extracting pupil data used for 
assigning teacher data of Interest to one of a plurality of classes, at least from a block other than a block corresponding 
to the teacher data of interest, with the processed data as teacher data as teacher in learning, and for outputting the 

so extracted data as class tap, finding a class of the teacher data of interest, based on the class tap, and effecting learning 
so that a prediction error of the prediction value of the teacher data obtained on predictive calculations using the class- 
based tap coefficients and pupil data will be statistically minimum to find the tap coefficients from class to class. 
[0049] According to the present Invention, converted data used in predictive calculations used for decoding data of 
a data block of Interest among data blocks Is extracted at least from a converted block corresponding to a data block 

S5 other than the data block of Interest for use as prediction taps. Using the tap coefficients and the prediction taps, preset 
predictive calculations are carried out to decode the converted data to original data. 

[0050] That is, the present Invention also provides a data processing apparatus including acquisition means for 
acquiring tap coefficients as found on learning, prediction tap extraction means for extracting the converted data used 
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for predictive calculations for decoding data of a data block of Interest among data blocks, each of which is a block ot 
the data, at least from a converted block as a block of the converted data, corresponding to a data block other than 
the data block of Interest, to output the extracted data as prediction tap, and calculating means for performing preset 
predictive calculations, using the tap coefficients and prediction tape, to decode the converted data to original data. 

5 [0051 ] The present Invention also provides a data processing method including the steps of acquiring tap coefficients 
as found on learning, extracting the converted data used for predictive calculations for decoding data of a data block 
of interest among data blocks each of which Is a block of the data, at least from a converted block as a block of the 
converted data, corresponding to a data block other than the data block of Interest, to output the extracted data as 
prediction tap, and performing preset predictive calculations, using the tap coefficients and prediction taps, to decode 

to the converted data to original data. 

[0052] The present Invention also provides a recording medium having recorded thereon a program including the 
steps of acquiring tap coefficients as found on learning, extracting the converted data used for predictive calculations 
for decoding data of a data block of interest among data blocks each of which Is a block of the data, at least from a 
converted block as a block of the converted data, corresponding to a data block other than the data block ot Interest, 
to output the extracted data as prediction tap, and performing preset predictive calculations, using the tap coefficients 
and prediction taps, to decode the converted data to original data. 

[0053] According to the present invention, pupil data as pupil are generated at least on orthogonal or frequency 
converting teacher data as teacher, and pupil data used in predictive calculations for decoding teacher data In a teacher 
block of interest among teacher blocks is extracted at least from a pupil block corresponding to a teacher block other 
20 than the teacher block of Interest for use as prediction tap. Tap coefficients are then found by effecting learning so that 
a predictive error of the prediction value of the teacher data obtained on performing predictive calculations using the 
tap coefficients and the prediction taps win be statistically minimum. 

[0054] That Is, the present Invention also provides a data processing apparatus Including generating means for 
generating pupil data as pupil, by at least applying orthogonal or frequency transform to teacher data as teacher, 

25 prediction tap extracting means for extracting the pupil data used for predictive calculations used for decoding teacher 
data of a teacher block of interest among teacher blocks each being a block of the teacher data, from at least a block 
of pupils as block of the pupil data corresponding to the teacher blocks other than the teacher block of Interest, to 
output the extracted data as predictive taps, and learning means for effecting learning so that a predictive error of the 
prediction value of the teacher data obtained on perfonning predictive calculations using the tap coefficients and the 

30 prediction taps win be statistically minimum. 

[0055] The present invention also provides a data processing method including the steps of generating pupil data 
as pupil, by at least applying orthogonal or frequency transform to teacher data as teacher, extracting the pupil data 
used for predictive calculations used for decoding teacher data ol a teacher block of Interest among teacher blocks, 
each being a block of the teacher data, from at least a block of pupils as block of the pupil data corresponding to the 

3? teacher blocks other than the teacher block of interest, to output the extracted data as predictive taps, and effecting 
learning so that a predictive error of the prediction value of the teacher data obtained on performing predictive calcu- 
lations using the tap coefficients and the prediction taps will be statistically minimum, to find tap coefficients. 
[0056] The present Invention also provides a recording medium having recorded thereon a program Including the 
steps of generating pupil data as pupil, by at least applying orthogonal or frequency transform to teacher data as 

40 teacher, extracting the pupil data used for predictive calculations used for decoding teacher data of a teacher block of 
Interest among teacher blocks each being a block of the teacher data, from at least a block of pupils as block of the 
pupil data corresponding to the teacher block other than the teacher block of Interest, to output the extracted data as 
predictive taps, and effecting learning so that a predictive error of the prediction value of the teacher data obtained on 
perfonning predictive calculations using the tap coefficients and the prediction taps will be statistically minimum to find 

45 the tap coefficients. 

[0057] According to the present Invention, using first data for learning of an extraction pattern and second data, the 
correlation of the first data showing a preset position relationship with respect to the second data lying In a preset 
position Is found from one preset position relationship to another. An extraction pattern of the first data to be used as 
the prediction tap used for the predictive calculations executed for obtaining the second data is set based on the so 
so found correlation. 

[0058] That Is, the present Invention also provides a data processing apparatus Including correlation calculating 
means for finding the correlation of the first data showing a preset position relationship with respect to second data 
lying In a preset position, the first and second data being used for learning the extraction pattern, from one the position 
relationship to another, and setting means for setting the extraction pattern of the first data to be used as the prediction 
55 tap used for the predictive calculations executed for obtaining the second data, based on the correlation. 

[0059] The present invention also provides a data processing method including the steps of finding the correlation 
of the first data showing a preset position relationship with respect to second data lying In a preset position, the first 
and second data being used for learning the extraction pattern, from one the position relationship to another, and setting 
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the extraction pattern of the first data to be used as the prediction tap used for the predictive calculations executed for 
obtaining the second data, based on the correlation. 

[00601 The present Invention also provides a recording medium having recorded thereon a program Including the 
step of finding the correlation of the first data showing a preset position relationship with respect to second data lying 
5 in a preset position, the first and second data being used for learning the extraction pattern, from one the position 
relationship to another, and setting the extraction pattern of the first data to be used as the prediction tap used for the 
predictive calculations executed for obtaining the second data, based on the correlation. 

Brief Description of the Drawings 

10 

[0O61] 

Flgs.i A, 1B and 1C Illustrate conventional JPEG encoding/decoding. 

Flg.2 shows an Illustrative structure of a picture transmission system embodying the present Invention. 
is Flg.3 Is a block diagram showing an Illustrative structure of a decoder In the picture transmission system. 

Fig. 4 Is a flowchart showing the processing In a decoder. 

Flg.5 Is a block diagram showing an Illustrative structure of a coefficient conversion circuit In the decoder. 
Flg.6 Illustrates Instances of a prediction tap and a class tap In the decoder. 

Fig.7 Is a block diagram showing an illustrative structure of the classification circuit in the coefficient conversion 
20 circuit 

Fig. 8 Illustrates the processing of a power calculation circuit In the coefficient conversion circuit. 

Fig.9 Is a flowchart for Illustrating the processing In the coefficient conversion circuit. 

Fig.10 Is a flowchart for Illustrating the processing In the coefficient conversion circuit In more detail. 

Flg.1 1 Is a block diagram showing an Illustrative structure of a learning device used for learning the tap coefficients 
25 in the coefficient conversion circuit. 

Flg.12 is a flowchart for Illustrating the processing of the learning device. 

Flg.13 Is a block diagram showing an Illustrative structure of the coefficient conversion circuit. 

Fig.14 Is a block diagram showing an Illustrative structure of the learning device. 

Fig.15 Is a block diagram showing another illustrative structure of the coefficient conversion circuit. 
30 Flg.1 6 is a block diagram showing still another Illustrative structure of the learning device. 

Fig. 17 is a block diagram showing still another illustrative structure of the coefficient conversion circuit. 

Fig. 1 8 Is a block diagram showing still another illustrative structure of the learning device. 

Flg.1 9 is a block diagram showing an Illustrative structure of an encoder In the picture transmission system. 

Flg.20 Is a block diagram showing an Illustrative structure of an MPEG decoder. 
35 Fig.21 Is a block diagram showing still another Illustrative structure of the coefficient conversion circuit. 

Flg.22 Is a block diagram showing still another Illustrative structure of the learning device. 

Fig.23 is a block diagram showing an illustrative structure of a decoder in another embodiment in which the present 

invention Is applied to the picture transmission system. 

Flg.24 shows how 9X8 DCT coefficients In the decoder are decoded to 1 6X 1 6 pixels. 
40 Rg.25 Is a flowchart for Illustrating the processing In the decoder. 

Fig.26 Is a block diagram showing an Illustrative structure of a coefficient conversion circuit in the decoder. 
Flg.27 Is a block diagram showing an Illustrative structure of a classification circuit In the coefficient conversion 
circuit 

Fig.28 Is a flowchart for Illustrating the processing In the coefficient conversion circuit... 
45 Flg.29 Is a flowchart for Illustrating the processing In the coefficient conversion circuit In more detail. 

Flg.30 Is a block diagram showing an Illustrative structure of a learning device responsible for learning processing 

of the tap coefficients of the coefficient conversion circuit. 

Flg.3 1 Is a flowchart for illustrating the processing in the learning device. 

Fig.32 is a block diagram showing another illustrative structure of the coefficient conversion circuit. 
so Flg.33 is a block diagram showing still another Illustrative structure of the learning device. 

Flg.34 Is a block diagram showing still another Illustrative structure of the coefficient conversion circuit. 

Flg.35 Is a block diagram showing still another Illustrative structure of the learning device. 

Flg.36 Is a block diagram showing still another illustrative structure of the coefficient conversion circuit. 

Flg.37 Is a block diagram showing still another Illustrative structure of the learning device. 
55 Fig.38 is a block diagram showing an illustrative structure of the encoder. 

Fig.39 is a block diagram showing still another illustrative structure of the learning device. . 

Flg.40 is a block diagram showing still another Illustrative structure of the learning device. 

Flgs.41 A and 41 B show a picture In which time resolution has been Improved. 
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Flgs.42A and 42B show a picture in which time resolution has been Improved. 

Rg.43 shows that shows that a class tap and a prediction tap are constructed from the DCT coefficients of two or 
more frames. 

Fig.44 is a block diagram showing an illustrative structure of a decoder in a embodiment In which the present 

Invention Is applied to the picture transmission system. 

Flg.45 Is a flowchart for Illustrating the processing by the decoder. 

Fig.46 Is a block diagram showing an illustrative structure of a coefficient conversion circuit in the decoder. 
Flg.47 is a block diagram showing an Illustrative structure of a classification circuit In the coefficient conversion 
circuit 

Flg.48 le a flowchart for Illustrating the processing by the coefficient conversion circuit 

Fig.49 Is a flowchart for illustrating the processing by the coefficient conversion circuit in more detail. 

Flg.50 Is a block diagram showing an Illustrative structure of a learning device learning tap coefficients of the 

coefficient conversion circuit. 

Flg.51 Is a flowchart for Illustrating the processing by the learning device. 

Fig.52 Is a block diagram showing another Illustrative structure of the coefficient conversion circuit. 

Flg.53 Is a block diagram showing an illustrative structure of a learning device. 

Flg.54 Is a block diagram showing another illustrative structure of the coefficient conversion circuit 

Fig. 55 is a block diagram showing an Illustrative structure of the learning device embocfylng the present Invention. 

Fig.56 is a block diagram showing another illustrative structure of the coefficient conversion circuit. 

Flg.57 is a block diagram showing an Illustrative structure of the learning device. 

Flg.58 Is a block diagram showing an Illustrative structure of an encoder In the picture transmission system. 
Fig.59 Is a block diagram showing an Illustrative structure of an MPEG decoder In the picture transmission system. 
Flg.60 is a block diagram showing another Illustrative structure of the coefficient conversion circuit. 
Hg.61 Is a block diagram showing another illustrative structure of the learning device. 

Fig. 62 is a block diagram showing an illustrative structure of a decoder in a embodiment in which the present 

Invention Is applied to the picture transmission system. 

Fig. 63 is a flowchart for Illustrating the processing of the decoder. 

Fig.64 Is a block diagram showing an Illustrative structure of the coefficient conversion circuit in the decoder. 

Flgs.65A and 65B Illustrate an Instance of a cluster tap in the coefficient conversion circuit. 

Flg.66 Is a block diagram showing an Illustrative structure of a classification circuit In the coefficient conversion 

circuit 

Flg.67 is a flowchart for Illustrating the processing of the coefficient conversion circuit. 

Flg.68 Is a flowchart for Illustrating the processing of the coefficient conversion circuit In more detail. 

Fig.69 Illustrates a classification method. 

Flg.70 is a flowchart for illustrating the processing of the classification circuit. 
Fig. 71 shows another method for classification. 

Fig. 72 is a block diagram showing another illustrative structure of the classification circuit. 

Flg.73 Is a flowchart for Illustrating the processing of the classification circuit. 

Fig.74 is a block diagram showing another Illustrative structure of the coefficient conversion circuit. 

Flg.75 Is a block diagram showing an Illustrative structure of a learning device responsible for learning processing 

of tap coefficients of the coefficient conversion circuit. 

Flg.76 Is a block diagram showing an illustrative structure of the coefficient conversion circuit. 
Fig.77 is a block diagram showing another illustrative structure of the learning device. 

Flg.78 is a block diagram showing an illustrative structure of a embodiment of a decoder In which the present 

Invention Is applied to the picture transmission system. 

Flg.79 is a flowchart for illustrating the processing by the decoder. 

Flg.BO is a block diagram showing an Illustrative structure of a coefficient conversion circuit In the decoder. 
Fig.81 is a block diagram showing an Illustrative structure of a classification circuit in the coefficient conversion 
circuit 

Rg.B2 is a flowchart for illustrating the processing by the coefficient conversion circuit 

Flg.83 Is a flowchart for Illustrating the processing by the coefficient conversion circuit In more detail. 

Flg.84 is a block diagram showing an Illustrative structure of a tap coefficient learning device responsl ble for learning 

processing of the pattern Information. 

Flg.B5 is a flowchart for illustrating the processing by the tap coefficient learning device. 

Fig.86 is a block diagram showing an illustrative structure of a pattern learning device responsible for learning 
processing of the pattern information. 

Flgs.B7A, 87B and 87C Illustrate the processing by an adder circuit In the pattern learning device. 
Flg.86 Is a flowchart for Illustrating the processing by the pattern learning device. 
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FIg.B9 ts a block diagram showing an Illustrative structure of the coefficient conversion circuit. 

Flg.90 Is a block diagram showing another Illustrative structure of the tap coefficient learning device. 

Flg.91 Is a block diagram showing another Illustrative structure of the pattern learning device. 

Flg.92 Is a block diagram showing another Illustrative structure of the coefficient conversion circuit. 
5 Fig. 93 Is a block diagram showing still another Illustrative structure of the tap coefficient learning device. 

Flg.94 Is a block diagram showing another Illustrative structure of the pattern learning device. 

Figs.95A, 95B and 95C Illustrate the processing of an adder circuit In the pattern learning device. 

Flg.96 Is a block diagram showing another Illustrative structure of the coefficient conversion circuit. 

Flg.97 Is a block diagram showing another illustrative structure of the tap coefficient learning device. 
10 Fig.98 Is a block diagram showing another illustrative structure of the pattern learning device. 

Fig.99 is a block diagram showing another illustrative structure of the coefficient learning device. 

Flg.100 Is a block diagram showing still another illustrative structure of the tap coefficient learning device. 

Fig.101 Is a block diagram showing still another illustrative structure of the pattern learning device. 

Flg.1 02 Is a block diagram showing illustrative structure of a embodiment of a computer embodying the present 
is Invention. 

Best Mode for Carrying out the Invention 

[0062] Referring to the drawings, the best mode for carrying out the present invention is explained in detail. 
20 [0063] The present Invention Is applied to a picture transmission system 20 configured for example In Flg.2. 

[0064] In this picture transmission system 20, picture data to be transmitted Is supplied to an encoder 21 . The encoder 

21 encodes the picture data transmitted thereto by e.g., JPEG encoding to fonn encoded data. That Is, the encoder 

21 Is configured similarly to a JPEG encoding device 5 shown In Fig.1 A and encodes picture data by JPEG encoding. 

The encoding data, obtained on J PEG encoding by the encoder 21 , Is recorded on a recording medium 23, such as a 
25 semiconductor memory, magnetooptical disc, a magnetic disc, an optical disc, a magnetic tape or a phase change 

disc, or Is transmitted over a transmission medium 24. such as ground waves, satellite network. CATV (cable television) 

network, Internet or public network. 

[0065] The decoder 22 receives encoded data transmitted from the recording medium 23 or over the transmission 
medium 24 for decoding to the original picture data. The so decoded picture data are supplied to a monitor, not shown, 
so for e.g., display. 

[0066] The decoder 22 includes an entropy decoding circuit 31 , a coefficient conversion circuit 32 and a deblocking 
circuit 33. as shown In Flg.3. 

[0067] The encoded data are sent to the entropy decoding circuit 31 , wh Ich then entropy-decodes the encoded data 
to send the resulting block-based quantized DCT coefficients Q to the coefficient conversion circuit 32. As explained 
35 in connection with the entropy decoding circuit 1 1 of Fig. I C, hot only the entropy encoded quantized DCT coefficients 
but atso the quantization table Is contained In the encoded data. This quantization table can be used as necessary for 
decoding the quantized DCT coefficients, as will be explained subsequently. 

[0068] Using the quantized DCT coefficients Q from the entropy decoding circuit 31 and tap coefficients as found by 
learning, as later explained, the coefficient conversion circuit 32 performs preset predictive calculations for decoding 
to the block-based quantized DCT coefficients to an original block of 8x8 pixels. 

[0069] The deblocking circuit 33 deblocks the decoded block, obtained in the coefficient conversion circuit 32, to 
produce an output decoded picture. 

[0070] Referring to a flowchart of Fig.4, the processing by the decoder 22 of Fig.3 is explained. 

[0071 ] The encoded data is sequentially supplied to the entropy decoding circuit 31 . At step S^\X\& entropy decoding 

45 circuit 31 entropy-decodes the encoded data to send block-based quantized DCT coefficients Q to the coefficient 
conversion circuit 32. The coefficient conversion circuit executes predictive calculations, using tap coefficients, to de- 
code the block-based quantized DCT coefficients Q from the entropy decoding circuit 31 into block-based pixel values, 
which then are sent to the deblocking circuit 33. The deblocking circuit 33 at step S3 executes deblocking of pixel 
values from the coefficient conversion circuit 32 to output the resulting decoded picture to finish the processing. 

so [0072] The coefficient conversion circuit 32 exploits the classification processing to decode the quantized DCT co- 
efficients Into pixel values. 

[0073] The classification adaptive processing Is comprised of a classification processing and an adaptive processing. 
By the classification processing, data is classified according to properties thereof and adaptive processing Is performed 
from class to class. This adaptive processing uses the following technique: 
55 [0074] That is, the adaptive processing finds estimated values of the original pixel by linear combination of. for ex- 
ample, quantized DCT coefficients and preset tap coefficients, to decode the quantized DCT coefficients to the original 
pixel values. 

[0075] Specifically, It is now contemplated that a certain picture Is teacher data, and another similar picture is DCTed 
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on the block basts and quantized to give quantized DCT coefficients as pupil data, and that an estimated value E[y] ol 
a pixel value y of a pixel ot the picture as teacher data Is found by a linear combination model prescribed by a linear 
combination of a set of a plural number of quantized DCT coefficients x 1 , X2, and preset tap coefficients , w 2 , 
[0076] In this case, the estimated value E[y] may be represented by the following equation (1 ): 

5 

E[y] = w 1 x 1 + w 2 x 2 + ... <1) 

[0077] If, for generalizing the equation (1), a matrix W, comprised of a set of tap coefficients Wj, a matrix X, comprised 
10 of a set of pupil data Xg and a matrix V comprised of a set of estimated values E[yJ are defined as 



13 



X = 



1 2 J 



30 



40 



Y'= 



50 the following observation equation (2) holds: 



XW = Y' 



(2) 



where a component x^ of the matrix X means number j pupil data of a number i pupil data set (a set of pupil data used 
for estimation of a number i teacher data y,) and a component Wj of the matrix Wis a tap coefficient the product of 
which with the number j pupil data in the pupil data set. On the other hand, y, denotes number i teacher data so that 
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E[yl] denotes an estimated value of the number I teacher data. Meanwhile, In y In the left side of the equation (1). a 
suffix I In the component V| of the matrix Y Is omitted, whereas, In , - ,n tne rt 9ht side of the equation (1 ), the suffix 
I In the component Xq of the matrix X Is omitted. 

[0078] It Is now contemplated to apply the least square method to this observation equation to find an estimated 
value Ely] approximate to the original pixel value y. If a matrix Y comprised of a set of true pixel values y as teacher 
data and a matrix E comprised of a set of residues e of the estimated value Ely] with respect to the pixel value y are 
defined as 



10 



13 



20 



25 



30 



yi 



3s the following residue equation (3): 

XW=Y+E (3) 

*o becomes valid from the equation (2). 

[0079] In this case, a tap coefficient Wj for finding the estimated value E[y] approximate to the original pixel value y 
may be found by minimizing the square error 



45 



[0080] Therefore, the tap coefficient wj for which the above square error differentiated with respect to the tap coef- 
ficient Wj Is zero, that Is the tap coefficient wj which satisfies the following equation (4): 

so 

dWj 1 dWj 1 dWj 

55 (/= 1,2,-.,J) 

represents an optimum value for finding the estimated value E proximate to the original pixel value y. 

[0081 J Thus, by differentiating the equatl on with respect to the tap coefficient Wj, the f ollowln g equation (5) Is obtain ed: 
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(5) 



5 (/=1>2 J) 

[0082] From the equations (4) and (5), the following equation (6) Is obtained: 



/ / / 

Z = *Z - °. -.Z e i*u - ° 



75 



-(6). 

[0083] If further the relation among pupil data Xg, tap coefficients w J( teacher data y, and the residues ej in the residue 
equation (3) Is considered, the following normal equation (7) Is obtained: 



20 



30 
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W • + <Z*n x u ) w j = (Z^i ) 

(Z**/ x nK + (Z 4 — 
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45 

-(7) 

[0084] Meanwhile, if the matrix (covariance matrix) A and a vector v are defined by 

55 
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45 respectively and the vector W Is defined as Indicated In the equation 1 , the normal equation shown In the equation (7) 
may be represented by the following equation (8): 



so 



AW=v 



(8) 



[0085] By providing a certain plural number of sets of the pupil data Xjj and sets of teacher data y h a number of normal 
equations of (7) equal to the number J of the tap coefficients v/j to be found may be established. Thus, by solving the 
equation (8) with respect to the vector W, an optimum number of tap coefficients (hero, the tap coefficients that minimize 
the square error) Wj may be found. It Is noted that, for solving the equation (8), the matrix A In the equation (8) needs 
to be regular. Meanwhile, a sweep-out method (Gauss-Jordan elimination method), for example, may be used. 
[0086] Finding an optimum tap coefficient wj as described above, and finding an estimated value E[y], proximate to 
the original pixel value y, by ihe equation (1). using the tap coefficient wj, represents adaptive processing. 
[0087] Meanwhile, If a picture which Is of the picture quality as a picture to be encoded by JPEG Is used as teacher 
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data, and quantized DOT coefficients obtained on DCTlng and quantizing the teacher data Is used as pupil data, such 
a tap coefficient Is obtained which will give a statistically minimum prediction error for decoding the JPEG encoded 
picture data to original picture data. 

[0088] Thus, even if the compression ratio at the time of performing J P EG encoding is raised, that is If the quantization 
s step used for quantization Is roughed, such decoding processing which statistically minimizes the prediction error may 
be realized by the adaptive processing, so that. In substance, the decoding processing of a JPEG encoded picture and 
the processing for Improving the picture quality may be performed simultaneously. As a result, the picture quality of 
the decoded picture may be maintained even If the compression ratio Is raised. 

[0089] If, for example, a picture of a higher quality than the picture to be JPEG encoded Is used as teacher data, 
to and quantized DCT coefficients, obtained on deteriorating the picture quality of the teacher data to the same picture 
quality as that of the picture to be JPEG encoded, followed by DCT and quantization, are used as pupil data, such tap 
coefficients are obtained which will statistically minimize the prediction error In decoding the JPEG encoded picture 
data to picture data of high picture quality. 

[0090] Thus, with the present adaptive processing, the decoding processing of the JPEG encoded picture and the 
is processing for improving the picture quality In this case may be performed simultaneously. Meanwhile, It may be seen 
from above that, by varying the picture quality of the picture which proves teacher or pupil data, it is possible to produce 
tap coefficients which will raise the picture quality of the decoded picture to an arbitrary level. 
[0091] Flg.5 shows an Illustrative structure of the coefficient conversion circuit 32 of Flg.3 which will decode the 
quantized OCT coefficients into pixel values by the above-described classification adaptive processing. 
20 [0092] in a coefficient conversion circuit 32A, shown in Flg.5, Mock-based quantized DCT coefficients, output by the 
entropy decodlngclrcult31 (Flg.3), are sent to a prediction tap extraction clrcult41 and to a class tap extraction circuit 42. 
[0093] The prediction tap extraction circuit 41 sequentially renders a block of pixel values corresponding to a block 
of the quantized DCT coefficients (incidentally referred to below as a DCT block), supplied thereto, a pixel block of 
Interest. It Is noted that the block of the pixel values, not present In the present stage but virtually presupposed, Is 
25 incidentally referred to below as a pixel block. The prediction tap extraction circuit 41 also sequentially renders respec- 
tive pixels making up the pixel block of Interest respective pixels of Interest In e.g., a raster scanning sequence. More- 
over, the prediction tap extraction circuit 41 extracts the quantized DCT coefficients used for predicting the pixel values 
of the pixels of Interest for use as predication taps. 

[0094] That is, the prediction tap extraction circuit 41 extracts the totality of the quantized DCT coefficients of a DCT 
30 block corresponding to the pixel block to which belongs the pixel of Interest, that is 8X8= 64 quantized DCT coefficients, 
as prediction taps, as shown in Rg.6. Thus, in the present embodiment, the same prediction taps are constituted for 
all pixels of a given pixel block. It should be noted that the prediction taps may also be formed by different quantized 
DCT coefficients from one pixel of Interest to another. 

[0095] The prediction taps for respective pixels forming a pixel block obtained in the prediction tap extraction circuit 
& 41 , that Is 64 sets of prediction taps for each of 64 pixels, are sent to a sum-of-products circuit 45. However, In the 
present embodiment, since the same prediction tap is formed for all pixels of one pixel block, it Is sufficient If one set 
of prediction taps is supplied to the sum-of -products circuit 45 for one pixel block. 

[0096] The class tap extraction circuit 42 extracts quantized DCT coefficients used for classifying the pixels of Interest 
Into one of plural classes, for use as class taps. 
40 [0097] Meanwhile, In JPEG encoding, since a picture Is encoded (DCTed and quantized) from one pixel block to 
another, the pixels belonging to one pixel block are classified In their entirety to, for example, the same class. Therefore, 
the class tap extraction circuit 42 forms the same class tap for respective pixels of a given pixel block. That Is, the class 
tap extraction circuit 42 extracts the totality of 8X8 quantized DCT coefficients of a DCT block, corresponding to the 
pixel block, to which belongs the pixel of interest, as a class tap. In the same way as in e.g., the prediction tap extraction 
circuit 41. 

[0098] It is noted that classifying the respective pixels belonging to a pixel block to the same class Is equivalent to 
classifying the pixel block. Thus, it is sufficient If the class tap extraction circuit 42 forme not 64 sets of the class taps 
for classifying each of 64 pixels making up the pixel block of interest, but one set of the class taps for classifying the 
pixel block of interest So, the class tap extraction circuit 42 extracts 64 quantized DCT coefficients of the DCT block 
so assod ated with the pixel block, in orderto classify the pixel bl ocks. from one pixel block to another, for use as class taps. 
[0O99] Meanwhile, the quantized DCT coefficients, forming prediction tap or class tap, are not limited to those of the 
above-mentioned patterns. 

[0100] The class taps of the pixel block of Interest, obtained In the class tap extraction circuit 42, are supplied to the 
classification circuit 43. The classification circuit classifies the pixel blocks of Interest, based on the class tap from the 
55 class tap extraction circuit 42, to output a class code corresponding to the so obtained class. 

[0101 ] As a classification method, ADRC (adaptive dynamic range coding), for example, may be used. 

[0102] With the method employing ADRC, the quantized DCT coefficients, forming the class tap, are processed with 

ADRC. The class of the pixel block of Interest Is determined based on the resultant ADRC code. 
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[0103] Meanwhile. In the K-blt ADRC, the maximum value MAX and the minimum value MIN of the quantized DCT 
coefficients forming the class taps are detected, and DR = MAX - MIN Is rendered a local dynamic range DR of a set. 
Based on this dynamic range DR, the quantized DCT coefficients, forming the class tap, are re-quantized to K bits. 
That le, the minimum value Ml N Is subtracted from the quantized DCT coefficients, forming the class tap, with the value 

5 resulting from the subtraction being then divided by DFV2K (quantization). The sequence of bits, obtained on arraying, 
In a preset order, the respective quantized K-blt DCT coefficients, forming the class taps, Is output as the ADRC code. 
Thus, If the class tap Is processed e.g., with 1-blt ADRC, the minimum value MIN is subtracted from the respective 
quantized DCT coefficients, forming the class taps, and the resulting difference Is then divided by a mean value of the 
maximum value MAX and the minimum value MIN, whereby the respective quantized DCT coefficients are made Into 

10 one-bit fonn, that is binary-coded. These 1 -bit quantized DCT coefficients are arrayed In a preset order to form a bit 
sequence which Is output as ADRC code. 

[01 04J The classification circuit 43 may output a pattern of level distribution of the quantized DCT coefficients, forming 
e.g., the class taps, directly as class codes. If, in this case, the class taps are formed by N quantized DCT coefficients, 
and K bits are allocated to the respective quantized DCT coefficients, the number of the class codes output by the 
is classification circuit 43 Is &") K , which Is an exorbitant figure exponentially proportionate to the number of bits K ofthe 
quantized DCT coefficlente. 

[0105J So, In the classification circuit 43, it is desirable to perform compression of the information of the class taps 
by the above-described ADRC processing or vector quantization, followed by classification. 

[0106] Meanwhile, in the present embodiment, the class taps are formed by the 64 quantized DCT coefficients, as 
20 described above. Thus, If the classification is executed by 1 -bit ADRC processing of the class taps, the number of the 
class codes Is a large figure of 2 s * 

[01071 Meanwhile, In the present embodiment, the classification circuit 43 extracts characteristic values of high cru- 
ciality from the quantized DCT coefficients forming the class taps and undertakes classification based on these char- 
acteristic values to diminish the number of the classes. 
25 [0108] That is, Fig.7 shows an illustrative structure of the classification circuit 43 of Fig.5. 

[0109] The class taps are adapted to be supplied to a power calculating circuit 51 . The power calculating circuit spirts 
the quantized DCT coefficients forming the class tap into a plural number of spatial frequency bands and calculates 
the power values of the respective frequency bands. 

[0110] That Is, the power calculating circuit 51 splits the 8x8 quantized DCT coefficients, forming the class taps, 

30 into tour spaclal frequency bands S 0 , S t , S 2 and S 3 , shown for example In Flg.8. 

[011 1 ] If now the 8X8 quantized DCT coefficients making up the class tap are represented with sequential Integral 
numbers from 0 In the raster scanning sequence, as shown In FIg.6, the spatial frequency band S 0 Is made up or four 
quantized DCT coefficients Xq, x-,, Xq, x 9 , the spatial frequency band S 0 Is made up of 12 quantized DCT coefficients 
*2, *3. x 4> x 5. *e» x 7 x io. X 12. x i3> X 14- *w. spatial frequency band S 0 Is made up of 1 2 quantized DCT coefficients 

35 x ie. X 17. x 24t *23> *&> x 40> X 41 . x 48» x 49» ^s. *57 and tn0 frequency band S 0 is made up of 36 quantized DCT 
coefficients x 18 , x 19 , x^, *2i , *22, x 2 3, x^, Xgy, x^, x^, x^, x 31 x^ x^, x^, x^, X30, x 39 X42, X43, X44, x^, x^, x 47 , 

XSO- x 51 • X S2» x 53- *54« x 55 x 58> X 5S- x 60» ^l* *62> ^ 

[0112] Moreover, the power calculating circuit 51 calculates power values P 0 , P 1t P 2 and P 3 of the AC components 
of the quantized DCT coefficients of the spatial frequency bands S 0 to S3 to output the so calculated power values P 0 , 

40 p i 1 p 2 and p 3 t0 a class code generating circuit 52. 

[0113] That is, the power calculating circuit 61 finds a square sum value x, 2 + x a 2 + Xg 2 of the AC components x v 
Xe, Xg of the four quantized DCT coefficients x^ x 1( Xgto output the resulting square sum to the class code generating 
circuit 52 as power P 0 . The power calculating circuit 51 also finds a square sum of the AC components of the 12 
quantized DCT coefficients for the spatial frequency band S 1( that is the totality of the 12 quantized DCT coefficients. 

43 to output the resulting square sum to the class code generating circuit 52 as power P t . The power calculating circuit 
51 also finds the power values P 2 and P 3 for the spatial frequency bands Sg and S 3l as In the case of the spatial 
frequency band S„ descrbed above, to output the resulting power values to the class code generating circuit 52. 
[0114] The class code generating circuit 52 compares the power values P 0 , P 1p P 2 and P 3 from the power calculating 
circuit 51 to corresponding threshold values TH 0 , TH^ TH 2 and TH3 stored in a threshold value storage unit 53 to 

so output class codes based on the magnitude relationships thereof. That Is, the class code generating circuit 52 compares 
the power P 0 to the threshold value TH 0 to produce a one-bit code representing the magnitude relationship, tn similar 
manner, the class code generating circuit 52 compares. the power P 1 to the threshold value TH 1( the power P 2 to the 
threshold value TH 2 and the power P 3 to the threshold value THg to produce a one-bit code for each of the power 
threshold value combinations. The class code generating circuit 52 outputs the four one-bit codes (thus any of values 

S5 from 0 to 1 5), obtained as described above, in e.g., a preset sequence, as class codes representing the classes of the 
pixel blocks of interest Thus, in the present embodiment, the pixel blocks of interest are classified into 2 4 = 16 classes. 
[01 15] The threshold value storage unit 53 memorizes the threshold values TH 0 to TH 3 to be compared to the power 
values P 0 to P 3 of the spatial frequency bands S 0 to S3. 
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[0116] In the above case, the DC component Xo of the quantized DCT coefficients Is not used for classification. 
However, the DC component x 0 may also be used for classification. 

[0117] Reverting to Fig.5, the class codes, output by the classification circuit 43, as described above, are afforded 
as addresses to a coefficient table storage unit 44. 

s [01 18] The coefficient table storage unit 44 memorizes a coefficient table, having registered therein tap coefficients 
obtained by learning processing, as will be explained subsequently, and outputs lap coefficients, stored in addresses 
associated with the class codes, output by the classification circuit 43, to the sum -of-pro ducts circuit 45. 
[0119] In the present embodiment, si nee the pixel blocks are classified, one class code is obtained for the pixel block 
of interest. On the other hand, since the pixel block in the present embodiment is made up of 8X8 = 64 pixels, 64 sets 

10 of tap coefficients are required for decoding 64 pixels making up a pixel block of Interest. Thus, the coefficient table 
storage unit 44 memorizes 64 sets of tap coefficients for an address corresponding to one class code. 
[0120] The sum-of-products circuit 45 acquires the prediction taps output by the prediction tap extraction circuit 41 
and the tap coefficients output by the coefficient table storage unit 44 and, using the prediction taps and the tap coef- 
ficients, executes linear predictive processing (sum of products processing) shown in the equation (1), to output the 

15 pixel values of the 8X8 pixels of the pixel block of interest as decoded results of the corresponding DCT block to the 
deblocking circuit 33 (Flg.3). 

[0121] In the prediction tap extraction circuit 41, each pixel of the pixel block of interest sequentially proves a pixel 
of Interest. The sum-of-products circuit 45 then performs processing in an operating mode associated with the position 
of a pixel which is currently the pixel of interest of the pixel block of interest. This operating mode Is incidentally referred 
20 to below as the pixel position mode. 

[0122] That Is, if a number i pixel of the pixels of the pixel block of Interest, In the raster scanning sequence, Is 
denoted p,, with the pixel p, being the pixel of Interest, the sum-of-products circuit 45 performs the processing of the 
pixel position mode #1. 

[0123] Specifically, the coefficient table storage unit 44 outputs 64 sets of tap coefficients for decoding each of the 
25 64 pixels making up a pixel block of interest. If the set of tap coefficients for decoding the pixel p, is denoted W fl and 
the operating mode Is the pixel position mode #1, the sum-of-products circuit 45 performs the sum of product processing 
of the equation (1), using the prediction taps and the set W h among the 64 sets of the tap coefficients, and renders the 
results of the sum of product processing the results of the decoding of the pixel p,. 

[0124] Referring to the flowchart of Rg.9, the processing of the coefficient conversion circuit 32A of Flg.5 is explained. 
30 [01 25] The block-based DCT coeff id ents , outp ut by the entropy decoding clrcu It 3 1 , are seq uentiai ly received by th e 
prediction tap extraction circuit 41 and by the class tap extraction circuit 42. The prediction tap extraction circuit 41 
sequentially renders the pixel block, corresponding to the block of the quantized DCT coefficients (DCT block), supplied 
thereto, a pixel block of interest 

[0126] At step S11 , the class tap extraction circuit 42 extracts the quantized DCT coefficients received and used for 
& classifying the pixel block of Interest, to form class taps, which then are output to the classification circuit 43. 

[0127] At step S1 2, using the class taps from the class tap extraction circuit 42, the classification circuit 43 classifies 
the pixel blocks of interest to output the resulting class codes to the coefficient tabla storage unit 44. 
[0126] That Is, at step S12 In the flowchart of Flg.1 0, the power calculating circuit 51 of the classification circuit 43 
(Fig. 7) splits, first at step S21 , the 8X8 quantized DCT coefficients, making up the class taps, into four spatial frequency 
40 bands S 0 to 5 3 to calculate the respective power values P 0 to P 3 . These power values P 0 to P 3 are output from the 
power calculating circuit 51 to the class code generating circuit 52. 

[0129] The class code generating circuit 52 at step S22 reads out the threshold values TH 0 to TH 3 to compare the 
power values P 0 to P 3 from the power calculating circuit 51 to the threshold values TH 0 to TH3 to generate the class 
codes based on the respective magnitude relationships. The program return then is performed. 
45 [0130] Reverting to Rg.9, the class codes, obtained at step S12 as described above, is given from the classification 
circuit 43 to the coefficient table storage unit 44 as address. 

[0131] On receipt of the class code from the classif ication circuit 43 as addresses, the coefficient table storage unit 
44 at step S13 reads out the 64 sets of the tap coefficients stored In the address to output the tap coefficients so read 
out to the sum-of-products circuit 45. 
50 [0132] The program then moves to step 514 where the prediction tap extraction circuit 41 extracts quantized DCT 
coefficients used for predicting the pixel values of the pixels of interest of the pixel block of interest not as yet rendered 
the pixels of interest in the raster scanning sequence to form prediction taps. These prediction taps are sent from the 
prediction tap extraction circuit 41 to the sum-of-products circuit 45. 

[0133] In the present embodiment, since the same prediction tap is formed for the totality of the pixels of the pixel 
55 block, from one pixel block to another, it is sufficient if the processing of step S1 4 is performed only for the pixel initially 
rendered a pixel of interest, while it is unnecessary to perform the processing for the remaining 63 pixels. 
[0134] At step 51 5, the sum-of-products circuit 45 acquires a set of tap coefficients corresponding to the pixel position 
mode for the pixel of interest, from among the 64 sets of the tap coefficients output at step S 13 from the coefficient 
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table storage unit 44 and, using the set of the tap coefficients and the prediction taps supplied from the prediction tap 
extraction circuit 41, the sum-of-products circuit 45 executes the sum of product processing, shown In the equation 
(1), to acquire a value of the decoded pixel of Interest. 

[0135] The program then moves to step S1 6 where the prediction tap extraction circuit 41 verifies whether or not the 
s processing has been executed with the totality of the pixels of the pixel block of Interest as the pixels of Interest. If It 
Is verified at step S16 that the processed has not been executed with the totality of the pixels of the pixel block ol 
interest as the pixels of Interest, the program reverts to step S1 4 where the prediction tap extraction circuit 41 repeats 
similar processing with the pixels of the pixel block of Interest not as yet rendered the pixels of Interest In the raster 
scanning sequence as the pixels of Interest. 
10 [0136] If It Is verified at step S 1 6 that the processing has been done with the totality of the pixels of the pixel block 
of Interest as the pixels of Interest, that Is if the decoded values of the totality of the pixels of the pixel block of interest 
have been acquired, the sum-of-products circuit 45 outputs the pixel block (decoded block) made up of the decoded 
values to the deblocking circuit 33 (Flg.3) to finish the processing. 

[0137] Meanwhile, the processing conforming to the flowchart of Flg.9 is executed repeatedly each time the prediction 
is tap extraction circuit 41 sets a new pixel block of Interest. 

[0138] Fig 11 shows an Illustrative structure of a learning device 60A executing the learning processing for tap co- 
efficients to be stored in the coefficient table storage unit 44 of Fig.5. 

[0139] A blocking circuit 61 Is fed with one or more pictures of learning picture data as teacher data serving as a 
teacher at the time of learning. The blocking circuit 61 blocks a picture as teacher data into pixel blocks each composed 
so of 8X8 pixels as In the case of the JPEG encoding. 

[0140] A DCT circuit 62 sequentially reads out pixel blocks, formed by the blocking circuit 61, as pixel blocks of 
Interest, and applies DCT processing to the pixel blocks of interest to render the pixel block of Interest a block of DCT 
coefficients. This block of DCT coefficients Is routed to a quantization circuit 63. 

[01 41 ] The quantization circuit 63 quantizes the blocks of the DCT coefficients from the DCT circuit 62 In accordance 
25 with the same quantization table as that used in JPEG encoding, and sequentially sends the resulting blocks of the 
quantized DCT coefficients (DCT blocks) to the prediction tap extraction circuit 64 and to the class tap extraction circuit 
42. 

[0142] The prediction tap extraction circuit 64 forms the same prediction taps as those formed by the prediction tap 
extraction circuit 41 of Fig. 5, for the pixel of interest among the pixels of the pixel block of interest which is as yet not 
30 rendered the pixel of Interest, In the raster scanning sequence, by extracting the needed quantized DCT coefficients 
from the output of the quantization circuit 63. These prediction taps are sent as pupil data as pupils for learning from 
the prediction tap extraction circuit 64 to the normal equation summation circuit 67. 

[0143] The class tap extraction circuit 65 forms the same class tap as that formed by the class tap extraction circuit 
42 of Flg.5, by extracting the needed quantized DCT coefficients from the output of the quantization circuit 63. This 
& class tap Is sent from the class tap extraction circuit 65 to the classification circuit 66. 

[0144] Using the class tap from the class tap extraction circuit 65, the classification circuit 66 performs the same 
processing as that of the classification circuit 43 of Fig.5 to classify the pixel blocks of interest to route the resulting 
class codes to a normal equation summation circuit 67. 

[0145] The normal equation summation circuit 67 reads out a pixel value of the pixel of Interest as teacher data from 
40 the prediction tap extraction circuit 64 and effects summation processing on the quantized DCT coefficients forming 
prediction taps as pupil data and on the pixel of interest. 

[0146] Thatls,thenomiaJequatlonsummatlonclr^ multiplication (x^J 

and summation £ of pupil data to pupil data, as respective components of the matrix A of the equation 8, using prediction 
taps (pupil data) from one class corresponding to the class code supplied from the classification circuit 66 to another. 
[0147] The normal equation summation circuit 67 also executes calculations corresponding to multiplication (x^,) 
and summation L of prediction taps (pupil data), and the pixel of Interest (teacher data), as components of the vector 
v of equation (8), using the prediction taps (pupil data) and the pixel of Interest (teacher data), from one class corre- 
sponding to the class code supplied from the classification circuit 66 to another. 

[0148] The aforementioned summation in the nonnal equation summation circuit 67 is performed for each class from 
50 one pixel position mode for the pixel of interest to another. 

[0149] The normal equation summation circuit 67 executes the above-described summation, with the totality of the 
pixels making up the teacher picture supplied to the blocking circuit 61 as pixels of Interest, whereby a normal equation 
shown In the equation (8) is established for each pixel position mode for each clase. 

[0150] A tap coefficient decision circuit 68 solves the normal equation, generated for each class, from one pixel 
55 position mode to another, in the nonnal equation summation circuit 67, to find 64 sets of the tap coefficients, from class 
to class, to send the tap coefficients thus found to the addresses corresponding to the respective classes of a coefficient 
table storage unit 69. 

[0151] Depending on the number of pictures provided as pictures for learning, or on the picture contents, It may be 
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an occurrence that act ass is produced In which the number of the normal equations necessary to find the tap coefficients 
cannot be obtained. For such class, the tap coefficient decision circuit 68 outputs e.g., default tap coefficients. 
[01 52] The coefficient table storage unit 69 memorizes 64 sets of the tap coefficients for each class supplied thereto 
from the tap coefficient decision circuit 68. 
s [0153] Referring now to the flowchart of Rg .1 2, the processing (learning processing) of a learning device 60A of Fig. 
11 Is explained. 

[0154] The blocking circuit 61 . fed with picture data for learning as teacher data, blocks the picture data as teacher 
data Into an 8X8 pixel block, as In the case of JPEG encoding. The program then moves to step S32 where the DCT 
circuit 62 sequentially reads out the pixel blocks, formed by the blocking circuit 61 , and applies DCT processing to the 
10 pixel block of interest to render the pixel block a block of DCT coefficients. The program then moves to step S33 where 
the quantization circuit 63 sequentially reads out the blocks of the DCT coefficients obtained for JPEG encoding to 
fonn blocks each made up of quantized DCT coefficients (DCT block). 

[0155] The program then moves to step S34 where the class tap extraction circuit 65 makes the pixel blocks blocked 
by the blocking circuit 61 and which are not as yet rendered pixel blocks of Interest Into pixel blocks of Interest. The 

'* class tap extraction circuit 65 also extracts the quantized DCT coefficients used for classifying the pixel blocks of 
interest from the DCT block obtained at the quantization circuit 63 to form a class tap which is routed to the classification 
circuit 66. As in the case explained with reference to the flowchart of Fig. 1 0, the classification circuit 66 at step S35 
classifies the pixel blocks of Interest, using the class tap from the class tap extraction circuit 65, and routes the resulting 
class codes to the normal equation summation circuit 67. The program then moves to step S36. 

£0 [0156] At step S36, the prediction tap extraction circuit 64 makes the pixels of the pixel block of interest which have 
not been made Into the pixels of Interest, In the raster scanning sequence, Into pixels of Interest, and extracts the 
needed quantized DCT coefficients from the output of the quantization circuit 63 to form the same prediction taps as 
those formed by the prediction tap extraction circuit 41 of Flg.5. The prediction tap extraction circuit 64 sends prediction 
taps for the pixels of Interest as pupil data to the normal equation summation circuit 67. The program then moves to 

25 step 537. 

[0157] At step S37, the normal equation summation circuit 67 reads out pixels of Interest, as teacher data, rrom the 
blocking circuit 61 , and executes the processing of summation of the matrix A of the equation 8 and the vector v, as 
described above, on the quantized DCT coefficients forming the prediction tap as pupil data and pixels of Interest as 
teacher data. This summation Is executed for each of the classes corresponding to the class codes from the clasetfi- 

30 cation circuit classification circuit 66 for each of the pixel position modes for the pixels of Interest 

[0158] The program then moves to step S38 where the prediction tap extraction circurt 64 verifies whether or not the 
summation has been done with the totality of the pixels of the pixel block of Interest as the pixels of interest If summation 
has not been done forthe totality of the pixels of the pixel block of Interest as the pixels of Interest, the program reverts 
to step S36 where the prediction tap extraction circuit 64 makes the pixels of the pixel block of interest not as yet 

36 rendered the pixels of Interest in the raster scanning sequence Into new pixels of Interest The above-described process- 
ing Is repeated. 

[0159] If it is verified at step S38 that the summation has been done with the totality of the pixels of the pixel block 
of Interest as the pixels of Interest, the program moves to step S39 where the blocking circuit 61 verifies whether or 
not the processing has been done with the totality of the pixel blocks obtained form the picture as teacher data as the 

*o pixel blocks of Interest. If It Is verified at step S39 that the totality of the pixel blocks obtained from the picture as the 
teacher data have not been proceesed as yet as the pixel blocks of interest, the program reverts to step S34 where 
the pixel blocks formed by the blocking circuit 61 and which have as yet not been made Into the pixel blocks of Interest 
are newly rendered the pixel blocks of interest. Then, similar processing is carried out repeatedly. 
[0160] if It is verified at step S39 that the totality of the pixel blocks obtained from the picture as teacher data have 

4* been processed as the pixel blocks of Interest, that Is if a normal equation for each pixel position mode has been 
obtained In the normal equation summation circuit 67 for each class, the program then moves to step S40 where the 
tap coefficient decision circuit 68 solves the normal equation generated from one pixel position mode of each class to 
another to find 64 sets of tap coefficients for each of respective 64 pixel position modes of each class to route the tap 
coefficients thus found to the address for each class of the coefficient table storage unit 69 for storage therein. The 

50 program then comes to a close. 

[0161] As described above, the tap coefficients forthe respective classes, stored In the coefficient table storage unit 
69, are stored In the coefficient table storage unit 44 of Flg.5. 

[0162] So, the tap coefficients stored In the coefficient table storage unit 44 have been found by effecting learning 
so that the prediction errors (herein the square errors) of the predicted values of the original pixel values, obtained on 
55 linear prediction calculations, will be statistical fy minimum. The result is that, with the coefficient conversion circuit 32A 
of Fig. 5, a JPEG encoded picture can be decoded to a picture limftlessly proximate to the original picture. 
[0163] Moreover, since the processing of decoding the JPEG encoded picture and the processing for Improving the 
picture quality are carried out simultaneously, a decoded picture of high picture quality can be efficiently produced from 



18 



EP 1 282 236 A1 



the JPEG encoded picture. 

[0164] Flg.13 shows another Illustrative structure oi the coefficient conversion circuit 32 of Flg.3. Meanwhile, parts 
or components similar to those shown In Fig .5 are depicted by the same reference numerals That Is, coefficient con- 
version circuit 32B shown in Flg.13 Is basically constructed similarly to the circuit shown In Rg.6 except that there Is 

s newly provided a dequantlzatlon circuit 71 . 

[0165] In the coefficient conversion circuit 32B, shown In Fig. 13, the dequantlzatlon circuit 71 Is fed with block-based 
quantized DCT coefficients, obtained on entropy decoding the encoded data In the entropy decoding circuit 31 (Flg.3). 
[0166] Meanwhile, In the entropy decoding circuit 31 , not only the quantized DCT coefficients but also the quantization 
table Is obtained form the encoded data. In the coefficient conversion circuit 32B of Fig. 19, this quantization table is 

10 supplied from the entropy decoding circuit 31 to the dequantlzatlon circuit 71 . 

[0167] The dequantlzation circuit 71 dequantlzes the quantized DCT coefficients from the entropy decoding circuit 
31 to send the resulting DCT coefficients to the prediction tap extraction circuit 41 and to the class tap extraction circuit 
42. 

[0168] Thus, In the prediction tap extraction circuit 41 and In the class tap extraction circuit 42, the prediction taps 
« and the class taps are formed for the DCT coefficients Instead of for the quantized DCT coefficients. The processing 
similar to that shown In Rg.6 is subsequently performed on the DCT coefficients. 

[0169] Thus, the coefficient conversion circuit 32B of Flg.13 performs processing not on the quantized DCT coeffi- 
cients but on the DCT coefficients, so that It Is necessary for the tap coefficients stored In the coefficient table storage 
unit 44 to be different from those shown in Fig. 5. 

zo [0170] Flg.14 shows another Illustrative structure of a learning device 60B adapted for learning the tap coefficients 
stored In the coefficient table storage unit 44 of F)g.13. The parts or components of Flg.14 corresponding to those 
shown In Flg.1 1 are Indicated by the same reference numerals and the corresponding explanation is sometimes omitted 
for clarity. That Is, the learning device 60B shown in Fig. 14 is basically configured similarly to the circuit shown in Fig. 
11 except that a dequantlzatlon circuit 81 is newly provided at back of the quantization circuit 63. 

25 [0171] In the learning device 60B on Rg.14. similarly to the dequarttization circuit 71 of Fig.13, the dequantization 
circuit 81 dequantlzes the quantized DCT coefficients, output by the dequantlzatlon circuit 63. The resulting DCT co- 
efficients are routed to the prediction tap extraction circuit 64 and to the class tap extraction circuit 65. 
[0172] Thus, in the prediction tap extraction circuit 64 class tap extraction circuit 65, not the quantized DCT coeffi- 
cients but the prediction taps and class taps are formed for the DCT coefficients. Subsequently, processing similar to 

so that shown In Flg.11 Is performed on the DCT coefficients. 

[0173] The result is that there are produced tap coefficients that diminish the effect of the quantization error resulting 
from quantization and dequantlzatlon of the DCT coefficients. 

[0174] Flg.15 shows a modified Illustrative structure of the coefficient conversion circuit 32 of Flg.3. The parts or 
components of Rg.15 corresponding to those shown In Rg.5 are Indicated by the same reference numerals and the 
35 corresponding explanation Is sometimes omitted for clarity. That Is, the coefficient conversion circuit 32C shown In Fig. 
13 Is basically configured similarly to the circuit shown In Flg.5 except that It Is destitute of the class tap extraction 
circuit 42 and the classification circuit 43. 

[0175] So. the coefficient conversion circuit 32C shown In Flg.15 lacks In the concept of the class, which may be 
said to be equivalent to the number of the classes being one. Thus, only one class ofthe tap coefficients Is stored In 
to the coefficient table storage unit 44 such that processing Is executed using this one class of the tap coefficients. 

[0176] Thai Is, in the coefficient conversion circuit 32C of Rg.15, the tap coefficients memorized In the coefficient 
table storage unit 44 differ from those shown In Rg.5. 

[0177] Fig.16 shows an illustrative structure of the learning device 60C adapted for executing the processing of 
learning the tap coefficients to be stored In the coefficient table storage unit 44 of Rg.15. Meanwhile, the parts or 
45 components of Flg.16 corresponding to those shown In Flg.1 1 are Indicated by the same reference numerals and the 
corresponding explanation Is sometimes omitted for clarity. That Is, the coefficient conversion circuit 60C shown In Fig. 

1 6 is basically configured similarly to the circuit ehown in Rg.11 except that It Is destitute of the class tap extraction 
circuit 65 and the classification circuit 66. 

[0178] Thus, in the learning device 60C of Rg.1 6, the above-described summation is carried out without dependency 
so on the class in the normal equation summation circuit 67. The tap coefficient decision circuit 6B solves the normal 
equations generated from one pixel position mode to another to find the tap coefficients. 

[0179] Flg.17 shows another Illustrative structure of the coefficient conversion circuit 32 of Rg.3. Meanwhile, the 
parts or components of Rg.17 corresponding to those ehown in Rgs.5 or 13 are Indicated by the same reference 
numerals and the corresponding explanation Is sometimes omitted for clarity. That is, the coefficient conversion circuit 
55 32D shown in Rg.1 7 is basically configured similarly to the circuit shown in Fig.5 except that it is destitute of the class 
tap extraction circuit 42 and the classification circuit 43 and that there is newly provided the dequantization circuit 71 . 
[0180] Thus, similarly to the coefficient conversion circuit 32C of Flg.15, the coefficient conversion circuit 32D of Fig. 

1 7 rhemorizes only the class coefficients of one ciass and processing Is carried out using these one-ciass tap coeffi- 
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dents. 

[0181] Moreover, In the coefficient conversion circuit 32D of Fig. 17, prediction taps are formed not on the quantized 
DCT coefficients but on the DCT coefficients output by the dequantlzatlon circuit 71, as In the coefficient conversion 
circuit 32B of Fig. 13. Subsequently, processing is further performed on the DOT coefficients. 

5 [0162] Thus, In the coefficient conversion circuit 32D, shown In Flg.1 7, the tap coefficients stored In the coefficient 
table storage unit 44 differ from those shown In Fig. 5. 

[0183] Thus, Fig. 1 B shows an illustrative structure of a learning device 60D executing the learning processing of tap 
coefficients to be stored in the coefficient table storage unit 44 of Flg.1 7. Meanwhile, the parts or components of Fig. 
1 8 corresponding to those shown In Figs. 1 1 or 1 4 are Indicated by the same reference numerals and the corresponding 
"> explanation is sometimes omitted for clarity. That is, the learning device 60D shown In Fig. 18 Is basically configured 
similarly to the learning device shown in Fig. 18 except that it is destitute of the class tap extraction circuit 65 and the 
classification circuit 66 and that there is newly provided the dequantlzatlon circuit 81 . 

[0184] Thus, In the learning device 6QD, shown in Flg.1 8. prediction taps are formed In the prediction tap extraction 
circuit 64 for the DCT coefficients, Instead of for the quantized DCT coefficients. Subsequently, processing is further 
i* performed on the DCT coefficients. Moreover, the normal equation summation circuit 67 executes the aforementioned 
summation without dependency on the class, such that the tap coefficient decision circuit 68 solves the normal equa- 
tions generated without dependency on the class to find the tap coefficients. 

[0185] Although the foregoing description has been made In connection with the Image produced on JPEG encoding 
designed for compression encoding a still image, it is also possible to process e.g., a picture obtained on MPEG 
20 encoding. 

[0186] That Is, Flg.19 shows an Illustrative structure of the encoder 21 when MPEG encoding Is performed. 
[0187] In the encoder 21 , shown In Fig.1 9, the frames or the fields, making up a moving picture to be encoded by 
MPEG, are sequentially sent to a motion detection circuit 91 and to a calculating unit 92. 

[0188] The motion detection circuit 91 detects the motion vector for the frame supplied thereto, on the macro-block 
25 basis, to send the so detected motion vector to an entropy decoding circuit 96 and to a motion compensation circuit 100. 
[0189] ir the picture supplied to the calculating unit 92 Is an l-plcture (Intra-plcture), the calculating unit 92 directly 
routes ft on the macro-block basis to the blocking circuit 93. If the picture supplied to the calculating unit 92 Is a P- 
picture (predictive coded picture) or a B-plcture (bidirectlonafty predicted picture), the calculating unit 92 calculates a 
difference from the reference picture supplied from the motion compensation circuit 1 00 to route the difference value 
so to the blocking circuit 93. 

[0190] The blocking circuit 93 blocks an output of the calculating unit 92 into a block of 8x8 pixels, which is supplied 
to a DCT circuit 94. The DCT circuit 94 applies DCT processing to the pixel block from the blocking circuit 93 to route 
the resulting DCT coefficients to a quantization circuit 95. The quantization circuit 95 quantizes the resulting Mock- 
based DCT coefficients in a preset quantization step to route the resulting quantized DCT coefficients to the entropy 

6 decoding circuit 96. The entropy decoding circuit 96 entropy-encodes the quantized DCT coefficients from the quan- 
tization circuit 95 to add the motion vector from the motion detection circuit 91 and other similar needed Information to 
output the resulting encoded data as MPEG encoded data. 

[0191] Of the quantized DCT coefficients, output by the quantization circuit 95. the l-and P-pictures need to be locally 
decoded 1or use as reference picture for subsequently encoded P- and B- Pictures. So, these pictures are routed not 
40 only to the entropy decoding circuit 96 but also to the dequantlzatlon circuit 97. 

[0192] The dequantlzatlon circuit 97 dequantizee the quantized DCT coefficients from the quantization circuit 95 Into 
DCT coefficients which are routed to an Inverse DCT circuit 98. The Inverse DCT circuit 98 applies Inverse DCT to the 
DCT coefficients from the dequantization circuit 97 to output the result to a calculating unit 99. This calculating unit 99 
is fed not only with the output of the Inverse DCT circuit 98 but also with a reference picture output by the motion 
compensation circuit 100. If the output of the Inverse DCT circuit 98 Is a P-plcture, the calculating unit 99 adds the 
output to an output of the motion compensation circuit 100 to decode the original picture which Is supplied to the motion 
compensation circuit 100. If the output of the Inverse DCT circuit 98 Is an l-plcture, the output Is the decoded l-plcture, 
ft is directly supplied to the motion compensation circuit 100. 

[0193] The motion compensation circuit 1 00 motion-compensates a locally decoded picture, supplied from the cal- 
5o cutating unit 99, In accordance with the motion vector from the motion detection circuit 91 , to route the motion-com- 
pensated picture as reference picture to the calculating unit 92 and to the calculating unit 99. 
[0194] Rg.20 shows an Illustrative structure of an MPEG decoder 110 for decoding encoded data obtained by the 
above-described MPEG encoding. 

[0198] In this MPEG decoder 1 1 0, the encoded data are sent to an entropy decoding circuit 1 1 1 . This entropy decoding 
55 circuit 111 entropy-decodes the encoded data to produce quantized DCT coefficients, motion vector and other infor- 
mation. The quantized DCT coefficients are sent to a dequantization circuit 112, whilst the motion vector is routed to 
a motion compensation circuit 116. 

[0196] The dequantlzatlon circuit 112 dequantizes the quantized DCT coefficients from the entropy decoding circuit 
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111 to form DCT coefficients which are supplied to an Inverse DOT circuit 113. 

[0197] The inverse DCT circuit 113 applies inverse DCT to the quantized DCT coefficients from the dequantization 
circuit 112 which are routed to a calculating unit 114. The calculating unitl 14is fed with both the output of the inverse 
DCT circuit 113 and with an output of the motion compensation circuit 11 6, as a reference picture, which Is an already 
5 decocted I- or P-plcture motion-compensated with a motion vector from the entropy decoding circuit 1 1 1 . If the output 
of the inverse DCT circuit 113 is a P- or B-plcture, the calculating unit 114 sums this output to an output of the motion 
compensation circuit 100 to decode an original picture which is routed to a deblocking circuit 115. If the output of the 
inverse DCT circuit 113 is an l-picture, the output is a decoded l-picture, so that It directly sends the output to the 
deblocking circuit 115. 

io [0198] The deblocking circuit 1 15 deblocks the decoded picture supplied thereto on the pixel block basis to produce 
and output a decoded picture. 

[0199] On the other hand, the motion compensation circuit 116 receives an l-plcture and a P-picture, among the 
decoded pictures output from the calculating unit 114. to process these pictures with motion compensation in accord- 
ance with the motion vector from the entropy decoding circuit 111. The motion compensation circuit 116 sends the 
15 motion compensated picture as reference picture to the calculating unit 114. 

[0200] The decoder 22 of Rg.3 Is able to decode the MPEG-encoded data efficiently Into a picture of high picture 
quality. 

[0201] That Is, encoded data Is sent to the entropy decoding circuit 31 which then entropy decodes the encoded 
data. The quantized DCT coefficients, motion vector and other information, obtained on this entropy decoding, are sent 

20 from the entropy decoding circuit 31 to the coefficient conversion circuit 32. 

[0202] Using the quantized DCT coefficients Q from the entropy decoding circuit 31 and tap coefficients as found on 
learning, the coefficient conversion circuit 32 executes preset predictive calculations, at the same time as It performs 
motion compensation In accordance with the motion vector from the entropy decoding circuit 31 , to decode the quan- 
tized DCT coefficients into original pixel values, which are routed to the deblocking circuit 33. 

25 [0203] The deblocking circuit 33 deblocks the pixel block of decoded pixels from the coefficient conversion circuit 32 
to produce and output a decoded picture. 

[0204] Fig.21 shows an illustrative structure of the coefficient conversion circuit 32 in case of decoding MPEG en- 
coded data by the decoder 22. Meanwhile, parts or components corresponding to those shown in Fig. 17 or 20 are 
depicted by the same reference numerals and are not explained specifically. That is, a coefficient conversion circuit 
30 32E, shown In Fig.21 , Is configured similarly to Fig. 17 except the re are provided the calculating unit 114andthe motion 
compensation circuit 116 downstream of the sum of products circuit 45 as shown in Fig.20. 

[0205] Thus, In the coefficient conversion circuit 32E, shown In Flg.21 , predictive calculations employing tap coeffi- 
cients are carried out in place of the inverse DCT processing by the inverse DCT circuit 113 shown in Fig.20. Subse- 
quently, a decoded picture is obtained in the same way as In Fig.20. 
35 [0206] Fig.22 shows an Illustrative structure of the learning device 60E adapted for learning tap coefficients to be 
stored In the coefficient table storage unit 44 of Flg.21 . Meanwhile, parts or components corresponding to those shown 
in Fig. 18 are depicted by the same reference numerals and are not explained specifically: 

[0207] In the learning device 60E, shown In Flg.22, a motion vector detection circuit 121 and a calculating unit 122 
are fed with a picture for learning as teacher data. The motion vector detection circuit 121, calculating unit 122, a 

40 blocking circuit 123, a DCT circuit 124, a quantization circuit 125, a dequantization circuit 127, an inverse DCT circuit 
128, a calculating unit 129 or a motion compensation circuit 130 perform the processing similar to that performed by 
the vector detection circuit 91 , calculating unit 92, a blocking circuit 93, a DCT circuit 94, a quantization circuit 95, a 
dequantization circuit 97, an inverse DCT circuit 98, a calculating unit 99 or a motion compensation circuit 100 of Fig. 
19, respectively, whereby quantized DCT coefficients similar to those output by the quantization circuit 95 of Fig.19 

45 are output from the quantization circuit 1 25. 

[0206] The quantized DCT coefficients, output by the quantization circuit 1 25, are routed to the dequantization circuit 
81 .which then dequantlzes the quantized DCT coefficients from the quantization circuit 125 Into DCT coefficients which 
are supplied to the prediction tap extraction circuit 64. This prediction tap extraction circuit 64 forms prediction taps . 
from the DCT coefficients from the dequantization circuit 61 to send the resulting prediction taps to the normal equation 

50 summation circuit 67. 

[0209] The normal equation summation circuit 67 executes the aforementioned summation on an output of the cal- 
culating unit 1 22 as teacher data and on the prediction taps from the dequantization circuit 81 as pupil data to generate 
the normal equations. 

[0210] The tap coefficient decision circuit 68 solves the normal equations, generated by the normal equation sum- 
55 mation circuit 67, to find tap coefficients, which then are sent to and stored In the coefficient table storage unit 69. 
[0211] Since the MPEG encoded data are decoded using the tap coefficients thus found, in the sum-of -products 
circuit 45, the decoding of the MPEG encoded picture and the processing for improving the picture quality may be 
performed simultaneously, whereby a decoded picture of high picture quality may be obtained from the MPEG encoded 
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picture. 

[0212] Meanwhile, the coefficient conversion circuit 32E may be formed wimout employing the dequantlzation circuit 

71 . In such case, the learning device 60E may be formed without providing the dequantlzation circuit 81 . 

[0213] The coefficient conversion circuit 32E on Rg.21 may also be designed using the class tap extraction circuit 

5 42 and the classification circuit 43, as in the case shown in Flg.5. 1 n such case, the learning device 60E may be designed 
with the class tap extraction circuit 65 and the classification circuit 66, as In the case of Fig. 11 . 

[0214] According to the present Invention, as described above, in which the tap coefficients as found on learning are 
acquired and used along with converted data to execute preset calculations to decode the converted data Into original 
data, the converted data can be decoded efficiently. 

"> [0215] Moreover, according to the present Invention, In which teacher data as a teacher is at least transformed 
orthogonally or Into a frequency domain to generate pupil data as pupils, and In which learning la executed to statistically 
minimize the prediction errors of the prediction values of the teacher data obtained on predictive calculations using the 
tap coefficients and the pupil data, In order to find tap coefficients, the orthogonal or frequency transformed data can 
be decoded efficiency using these tap coefficients. 

15 [0216] A modified embodiment of the present Invention Is hereinafter explained. 

[0217] In the modification, now explained, a decoder 222 composed of an entropy decoding circuit 231 , a coefficient 
decoding circuit 232 and a deblocking circuit 233, as shown in Fig. 23 , is used as the decoder 22 shown In Fig .2 to 
decode the encoded data. 

[0218] The encoded data is fed to the entropy decoding circuit 231 , which then entropy decodes the encoded data 
20 to send the block-based quantized OCT coefficients Q to the coefficient decoding circuit 232. Meanwhile, although not 
only the entropy-encoded quantized DCT coefficients but also the quantization table Is Included In the encoded data, 
this quantization table may be used as necessary for decoding the quantized DCT coefficients. 
[0219] Using the quantized DCT coefficients Q from the entropy decoding circuit 231 and tap coefficients as found 
by learning, the coefficient decoding circuit 232 executes preset predictive calculations to obtain data which correspond 
25 to block-based quantized DCT coefficients decoded into an original 6x8 pixel block and which has been processed in 
a manner of Improving the Improving the picture quality of the original block. That Is, while the original block Is made 
up of 8x8 pixels, the coefficient decoding circuit 232 yields a block composed of 16X16 pixels, having double the 
horizontal and vertical spatial resolution of the 8x8 pixels, as a result of the predictive calculations employing the tap 
coefficients. Here, the coefficient decoding circuit 232 decodes the block composed of 8X8 quantized DCT coefficients 
30 into a block composed of 1 6x 1 6 pixels, and outputs the latter block. 

[0220] The deblocking circuit 233 deblocks the 16x16 pixel block, obtained in the coefficient decoding circuit 232, 
to frnprove the spatial resolution , to output the resulting decoded picture. 

[0221 ] Referring to the flowchart of Flg.25, the processing of the decoder 222 of Flg.23 Is explained. 

[0222] The encoded data is sequentially supplied to the entropy decoding circuit 231 . At step S 1 01 , the entropy 

6 decoding circuit 231 entropy-decodes the encoded data, and sends block-based quantized DCT coefficients Q to the 
coefficient decoding circuit 232, which coefficient decoding circuit executes predictive calculations, employing tap co- 
efficients, at step S102, to generate a high-resolution block, which is on decoding the block-based quantized DCT 
coefficients Q from the entropy decoding circuit 231 Into block-based pixels values, and which Is Improved In the block 
spatial resolution. The high-resolution bloc, thus produced, Is sent to the deblocking circuit 233. The deblocking circuit 

40 233 at step S103 executes deblocking of the block of the Improved spatial resolution from the coefficient decoding 
circuit 232 to output the resulting high- resolution decoded picture to terminate the processing. 
[0223] The coefficient decoding circuit 232 of Flg.23 exploits the above-described classification adaptive processing 
to decode the quantized DCT coefficients into pixel values to yield a picture improved in spatial resolution. 
[0224] Fig. 26 shows an Illustrative structure of the coefficient decoding circuit 232 of Flg.23 adapted for decoding 

« the quantized DCT coefficients Into pixel values. 

[0225] In a coefficient decoding circuit 232 A, shown In Flg.26, the block-based quantized DCT coefficients, output 
by the entropy decoding circuit 231 (Flg.23), are routed to a block-based prediction tap extraction circuit 241 and to a 
class tap extraction circuit 242. 

[0226] The prediction tap extraction circuit 241 sets the blocks of pixel values of high picture quality corresponding 
so to the blocks of 8xb quantized DCT coefficients supplied thereto. Incidentally referred to below as DCT blocks, se- 
quentially as high picture quality blocks of Interest. These blocks of pixel values of high picture quality are not present 
In the present stage but virtually presupposed, and are Incidentally referred to below as high picture quality blocks. 
Moreover, the pixels that make up the high picture quality block of Interest are sequentially made into pixels of interest 
in the raster scanning sequence. In summation, the prediction tap extraction circuit 241 extracts the quantized DCT 
ss coefficients, used for predicting the pixel values of the pixels of interest, for use as prediction taps. 

[0227] That Is. the prediction tap extraction circuit 241 extracts, as prediction taps, the totality of the quantized DCT 
coefficients, that Is 8X8 = 64 quantized DCT coefficients, of the DCT block corresponding to the high picture quality 
block to which belongs the pixel of interest, as shown In Flg.6. Thus, In the present embodiment, the same prediction 
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tap Is formed Tor all pixels of a certain high picture quality block. Alternatively, the prediction tap may be formed by 
different quantized DOT coefficients from one pixel of Interest to another. 

[0228] The prediction taps of the respective pixels making up a high picture quality block, that is 256 sets of the 
prediction taps for each of the set of 1 6X6 = 256 pixels, obtained In the prediction tap extraction circuit 241 , are sent 
s to a sum-of-products circuit 245. However, since the same prediction taps are formed for all pixels of a high picture 
quality block. It Is In effect sufficient If a set pf the predication taps is provided to the sum-of-products circuit 245 for 
one high picture quality block. 

[0229] The class tap extraction circuit 242 extracts quantized DCT coefficients used for classifying the pixels of 
Interest Into one of a plural number of classes, for use as class taps. 

10 [0230] In JPEG encoding, a picture is encoded, that is DCTed and quantized, from one 8x8 pixel block to another. 
This 8X8 pixel block is incidentally referred to below as pixel block. Thus, the pixels belonging to a high picture quality 
block corresponding to a pixel block, Improved to high picture quality, are classified e.g.. Into the sameciass. That Is. 
similarly to the prediction tap extraction circuit 241. the class tap extraction circuit 242 extracts the totality of 8X8 
quantized DCT coefficients of a DCT block corresponding to the high picture quality block to which belongs the pixel 

is of Interest, shown In Flg.6, as class taps. 

[0231 ] It ie noted that assigning the respective pixels belonging to a high picture quality pixel block to the eame class 
is equivalent to classifying the pixel block. Thus, it Is sufficient If the dass tap extraction circuit 242 1orms not 256 sets 
of the class taps for classifying each of 1 6x1 6 = 256 pixels making up the high picture quality pixel block of Interest, 
but one SBt ol the class taps for classifying the high picture quality pixel block of interest. To this end. the class tap 

20 extraction circuit 242 extracts 64 quantized DCT coefficients of the DCT block associated with the high picture quality 
pixel block, In order to classify the high picture quality pixel blocks, from one high picture quality pixel block to another, 
for use as class taps. 

[0232] Meanwhile, the quantized DCT coefficients, forming prediction taps or class taps, are not limited to those of 
the above-mentioned patterns. 

25 [0233] The class taps of the high quality pixel block of interest, obtained in the class tap extraction circuit 242, are 
supplied to the classification circuit 243. The classification circuit classifies the high picture quality block of Interest, 
based on the class lap from the dass tap extraction circuit 242, to output a class code corresponding to the so obtained 
class. 

[0234] As a classification method, ADRC, for example, may be used. With the method employing AQRC, the quan- 
go tlzed DCT coeffldents, forming the dass taps, are processed with ADRC. The class of the pixel block of Interest Is 
determined based on the resultant ADRC code. 

[0235] Meanwhile. In the present embodiment, the classification circuit 243 extracts characteristic values of high 
cruclallty from the quantized DCT coefficients forming the class taps and undertakes classification based on these 
characteristic values to diminish the number of the classes. 
35 [0236] Fig .27 shows an illustrative structure of the classification circuit 243 of Fig .26. 

[0237] The class taps are adapted to be supplied to a power calculating circuit 251 . The power calculating drcult 
251 splits the quantized DCT coefficients forming the class taps into a plural number of spatial frequency bands and 
calculates the power values of the respective frequency bands. 

[0238] That Is, the power calculating circuit 51 splits the 8X8 quantized DCT coefficients, forming the class tap, Into 
40 four spaclal frequency bands S 0 , S 1( and S 3 , shown for example In Flg.6. The spatial frequency band S 0 Is made 
up of four quantized DCT coefficients x 0 , x lP Xg, Xg, the spatial frequency band is made up of 12 quantized DCT 
coefficients x 2 , ^ x* Xj, x 6> x 7 x 1t) , x 11( x 12 , x 13 , >q 4l x 15 , the spatial frequency band Sg is made up ol 12 quantized 
DCT coefficients x 16 , x 17 , x^, x^ s% x^, X33, x^, x^. x^, x^. x 56 . x s7 and the spatial frequency band S 3 is made up of 
36 quantized DCT coeffldents x 18 , x 19 . x^q. x 21 , x 22. x 23. x 26. *2 7 . x 2e. x ». *aa* X 31 x 34» x 3s- *qs> x 37. x 38. X42. x 43. 

43 K45, x 46» X47. Xso, X 51 , X52, X53, X 54) X 55 Xqq, Xsg, X^,, X^ , X62, x 63* 

[0239] Moreover, the power calculating circuit 251 calculates power values P 0 , Pi . P2 and p 3 of tne AC components 
of the quantized DCT coefficients, for each of the special frequency bands Sq, S, , S 2 and S3, to output the so calculated , 
power values P 0) P u P 2 and P 3 to a class code generating circuit 252. 

[0240] That is. the power calculating drcuit 251 finds a square sum value x n 2 + x 8 2 + Xg 2 of the AC components x 1 , 
so xg, X9 of the four quantized DCT coeffldents x<>, x 1( Xq, x g to output the resulting square sum to the dass code generating 
circuit 52 as power P 0 . The power calculating circuit 251 also finds a square sum of the AC components of the 12 
quantized DCT coefficients for the spatial frequency band that Is the totality of the 12 quantized DCT coefficients, , 
to output the resulting square sum to the class code generating drcult 252 as power P, . The power calculating drcult 
251 also finds the power values P 2 and P 3 for the spatial frequency bands Sg and S 3 , respectively, as In the case of: 
55 the spatial frequency band S 1f described above, to output the resulting power values to the dass code generating 
circuit252. 

[0241J The dass code generating circuit 252 compares the power values P 0 , P«t, P2 and P 3 from the power calculating 
circuit 251 to corresponding threshold values TH 0 , TH.,, TH 2 and TH 3 stored In a threshold value storage unit 253 to 
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output class codes based on the magnitude relationships thereof. That is, the class code generating circuit 252 com- 
pares the power P 0 to the threshold value TH 0 to produce a one-bit code representing the magnitude relationships. In 
similar manner, the class code generating circuit 252 compares the power P 0 to the threshold value TH 0 to produce a 
one-bit code representing the magnitude relationships thereof. Likewise, the class code generating circuit 252 com- 

s pares the power P 1 to the threshold value TH 1p the power P 2 to the threshold value TH 2 and the power P 3 to the 
threshold value TH 3 to produce a one-bit code for each of the power-threshold value combinations. The class code 
generating circuit 252 outputs the four one-bit codes (thus any of values from 0 to 15), obtained as described above, 
In e.g., a preset sequence, as class codes representing the classes of the pixel block of interest. Thus, In the present 
embodiment, the pixel blocks of Interest are classified Into 2* = 1 6 classes. 

10 [0242] The threshold value storage unit 253 memorizes the threshold values TH 0 toTH 3 to be compared to the power 
values P 0 to P 3 of the spatial frequency bands S 0 to 

[0243] Reverting to Fig. 26, the class codes, output by the classification circuit 243, as described above, are afforded 
as addresses to a coefficient table storage unit 244. 

[0244] The coefficient table storage unit 244 memorizes a coefficient table, having registered therein tap coefficients 
1* obtained by learning processing as will be explained subsequently, and outputs tap coefficients stored in an address 
associated with the class code, output by the classification circuit 243, to the sum-of -products circuit 245. 
[0245] In the present embodiment, one class code is obtained for the pixel block of interest. On the other hand, since 
the high picture quality pixel block In the present embodiment is made up of 16x16 = 256 pixels, 256 sets of tap 
coefficients are required for decoding 256 pixels making up a pixel block of interest. Thus, the coefficient table storage 
20 unit 244 memorizes 256 sets of tap coefficients for an address corresponding to one class code. 

[0246] The sum-of-pro ducts circuit 245 acquires the prediction tap output by the prediction tap extraction circuit 241 
and the tap coefficients output by the coefficient table storage unit 244 and, using the prediction taps and the tap 
coefficients, executes linear predictive processing (sum of products processing) shown In the equation (1), to output 
the prediction values of the pixel values of the 16x16 pixels of the pixel block of interest as decoded results of the 
as corresponding DCT block to the deblocking circuit 233 (Fig.23). 

[0247] In the prediction tap extraction circuit 241 , each pixel of the pixel block of Interest sequentially becomes a 
pixel of Interest. The sum-of -products circuit 245 then performs processing In an operating mode associated with the 
position of a pixel which Is the current pixel of Interest of the pixel block of Interest. This operating mode Is referred to 
below as pixel position mode. 

so [0248] That Is, If a number I pixel of the pixels of the pixel block of Interest, In the raster scanning sequence, Is 
denoted p,, with the pixel p, being the pixel of interest, the sum-of-products circuit 245 performs the processing of the 
pixel position mode #1. 

[0249] Specifically, the coefficient table storage unit 44 outputs 256 sets of tap coefficients for decoding each of the 
256 pixels making up a pixel block of interest. If the set of tap coefficients for decoding the pixel p, is denoted W,, and 
36 the operating mode is the pixel mode #1 > the sum-of-products circuit 425 performs the sum of product processing of 
the equation (1 ), using the prediction taps and the set W,, among the tap coefficients of the 64 sets, and renders the 
results of the sum of product processing the results of the decoding of the pixel Df. 

[0250] Referring to the flowchart of Flg.28. the processing ofthe coefficient conversion circuit 232A of Flg.28 Is ex- 
plained. 

40 [0251] The block-based DCT coefficients, output by the entropy decoding circuit 231 (Flg.23), are sequentially re- 
ceived by the prediction tap extraction circuit 241 and by the class tap extraction circuit 242. The prediction tap extraction 
circuit 241 sequentially renders the pixel block corresponding to the block of the quantized DCT coefficients supplied 
thereto (DCT block) a pixel block of interest. 

[0252] At step S1 1 1 , the class tap extraction circuit 242 extracts the quantized DCT coefficients received and used 
4* for classifying the pixel block of Interest, to form a class tap, which then Is output to the classification circuit 243. 

[0253] At step 5112, the classification circuit 243 classifies the high quality pixel block of Interest, using the class 
taps from the class tap extraction circuit 242, to output the resulting class codes to the coefficient table storage unit 244. 
[0254] That is, st step S11 2 In the flowchart of Fig.29, the power calculating circuit 251 of the classification circuit 
243 (Fig.27) splits, first at step S121 , the 8x8 quantized DCT coefficients making up the class taps into four spatial 
so frequency bands So to S3 to calculate the respective power values P 0 to P 3 . These power values P 0 to P 3 are output 
from the power calculating circuit 251 to the class code generating circuit 252. 

[0255] The class code generating circuit 252 at step S 122 reads out the threshold values TH 0 to TH 3 from the power 
calculating circuit 51 to compare the power values P 0 to P 3 from the power calculating circuit 251 to the threshold 
values THo to TH 3 to generate the class codes based on the respective magnitude relationships. The program return 
55 then is performed. 

[0256] Reverting to Fig. 28. the class codes, obtained at step S112 as described above, are given from the classifi- 
cation circuit 243 to the coefficient table storage unit 244 as addresses. 

[0257] On receipt of the class codes form the classification circuit 243 as addresses, the coefficient table storage 
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unit 244 at step S113 reads out the 256 sets of the tap coefficients stored In the addresses (256 sets of tap coefficients 
corresponding to the classes of the class codes) to output the tap coefficients so read out to the sum-of-products circuit 
245. 

[0258] The program then moves to step S 1 1 4 where the prediction tap extraction circuit 241 extracts quantized DCT 
s coefficients used for predicting the pixel values of the pixels of Interest, with the pixels of the pixel block of Interest not 

as yet rendered the pixels of Interest In the raster scanning sequence as the pixels of Interest, to form prediction taps. 

Thee prediction taps are sent from the prediction tap extraction circuit 41 to the sum-of-products circuit 245. 

[0259] in the present embodiment, since the same prediction taps are formed for the totality of the pixels of the high 

quality pixel blocks,from one high quality pixel block to another, it Is sufficient If the processing ofstepS1 14 Is performed 
10 only for the pixel initially made into a pixel of interest, while it Is unnecessary to perform the processing forthe remaining 

255 pixels. 

[0260] At step S115, the sum-of-products circuit 245 acquires a set of tap coefficients corresponding to the pixel 
position mode for the pixel of Interest, from among the 256 set tap coefficients output at stelp S113 from the coefficient 
table storage unit 244 and, using the set of the tap coefficients and the prediction taps supplied from the prediction tap 
« extraction circuit 241 , the sum-of-products circuit 245 executes the sum of product processing, shown In the equation 
(1), to acquire a value of the decoded pixel of interest. 

[0261 ] The program then moves to step S1 1 6 where the prediction tap extraction circuit 241 verifies whether or not 
the processing has been executed with the totality of the pixels of the high quality pixel block of Interest as the pixels 
of interest. If it is verified at step S1 1 6 that the processed has not been executed with the totality of the pixels of the 
20 high quality pixel block of Interest as the pixels of Interest, the program reverts to step S1 1 4 where the prediction tap 
extraction circuit 241 repeats similar processing with the pixel value of the pixel of Interest of the pixels of the high 
quality pixel block of Interest, not as yet rendered the pixels of Interest In the raster scanning sequence, as the pixels 
of Interest 

[0262] If It Is verified at step S1 1 6 that the processing has been done with the totality of the pixels of the high quality 
25 pixel block of interest as the pixels of Interest, that is If the decoded values of the totality of the pixels of the high quality 

pixel block of Interest, that Is 8x8 quantized DCT coefficients decoded Into 8X8 pixels and further made Into 1 6x16 

pixels byway of Improvlngthe pixel quality, have been acquired, the sum-of-products circuit 245 outputs the high quality 

block made up of the decoded values to the deblocking circuit 233 (Flg.23) to finish the processing. 

[0263] Meanwhile, the processing conforming to the flowchart of Flg.28 is executed repeatedly each time the pre- 
30 diction tap extraction circuit 241 sets a new high quality pixel block of Interest. 

[0264] Fig!30 shows an illustrative structure of a learning device 270A executing the learning processing for tap 

coefficients stored In the coefficient table storage unit 244 on Flg.26. 

[0265] In the learning device 270A, a decimation circuit 260 Is fed with one or more pictures of picture data for 
learning, as teacher data which becomes a teacher In learning. The decimation circuit 260 processes the picture, as 

35 teacher data, for improving the picture quality of the picture as teacher data by sum of products processing performed 
by the sum-of-products circuit 245 In the coefficient decoding circuit 232A of Flg.26. Since this processing converts 
the 8XB pixels into 1 6x 1 6 pixels of the spatial resolution, along the horizontal and vertical directions, twice that of the 
8xB pixels, the decimation circuit 260 decimates the pixels of the picture data as teacher data to generate picture data, 
having the numbers of the horizontal and vertical pixels reduced to one half. This picture data Is Incidentally referred 

40 to below as quasl-teacher data. 

[0266] The picture data as quasl-teacher data Is of the same picture quality (resolution) as thee picture data to be 
encoded by JPEG encoding In the encoder 21 (Flg.2). For example, If the picture to be JPEG-encoded Is an SD. 
(standard resolution) picture, It is necessary to use a high density (HD) picture, having the number of the horizontal 
and vertical pixels twice those of the SD picture, as the picture used as teacher data. 

45 [0267] A blocking circuit 261 blocks an SD picture, as one or more pictures of quasl-teacher data, generated by a 
decimation circuit 260, Into an 8x8 pixel block, as In the case of JPEG encoding. 

[0268] A DCT circuit 262 sequentially reads out a pixel block, blocked by the blocking circuit 261 , and applies DCT 
to the pixel block to form a block ol DCT coefficients. This block of DCT coefficients is routed to a quantization circuit 263. 
[0269] The quantization circuit 263 quantizes the block of DCT coefficients from the DCT circuit 262 in accordance 
so with the same quantization table as that used for JPEG encoding In the encoder 21 (Flg.2), and sequentially sends 
the resulting block of the quantized DCT coefficients (DCT block) to a prediction tap extraction circuit 264 and to a 
class tap extraction circuit 265. 

[0270] The prediction tap extraction circuit 264 forme the same prediction taps as those formed by the prediction tap 
extraction circuit 241 of Flg.26, forthe pixels as current pixels of interest, among the 16X16 pixels making up the high 
55 picture quality block, which a normal equation summation circuit 267 retains to be the high picture quality block of 
interest, by extracting the needed quantized DCT coefficients from the output of the quantization circuit 263. These 
prediction taps are routed as pupil data, which become pupils In learning, from the prediction tap extraction circuit 264 
to the normal equation summation circuit 267. 
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[0271 ] The class tap extraction circuit 265 forms the same class taps as those formed by the class tap extraction 
circuit 242 of Flg.26, for the high picture quality block of Interest, by extracting the needed quantized DCT coefficients 
from the output of the quantization circuit 263. These class taps are supplied from the class tap extraction circuit 265 
to the classification circuit 266. 
5 [0272] The classification circuit 266 performs the same processing as that by the classification circuit 243, using the 
class taps from the class tap extraction circuit 265, to classify the high picture quality blocks of Interest to send the 
resulting class code to the normal equation summation circuit 267. 

[0273] The normal equation summation circuit 267 Is fed with the same HD picture as that supplied as teacher data 
to the decimation circuit 260. The normal equation summation circuit 267 blocks the HD picture Into 16x16 pixel high 
10 picture quality blocks which are sequentially made Into high picture quality blocks of Interest. The normal equation 
summation circuit 267 sequentially sets pixels, among the 16x16 pixels forming the high picture quality block of interest, 
which have as yet not been made Into the pixels of interest, as pixels of Interest, and executes summation on the pixel 
values of the pixels of Interest and the quantized DCT coefficients forming the prediction tap from the prediction tap 
forming circuit 264. 

[0274] That Is, the normal equation summation circuit 267 executes calculations corresponding to multiplication 
(XjnXj,,,) and summation £ of pupil data, as respective elements of the matrix A of the equation 8, using prediction tape 
(pupil data) from one class corresponding to the class code supplied from the classification circuit 266 to another 
[0275] The normal equation summation circuit 267 also executes calculations corresponding to multiplication (x,^ 
and summation E of prediction taps (pupil data), and the pixel of interest (teacher data), as components of the vector 
£0 v of equation (8), using the prediction taps (pupil data) and the pixel of Interest (teacher data), from one class corre- 
sponding to the class code supplied from the classification circuit 266 to another. 

[0276] The aforementioned summation in the normal equation summation circuit 267 is performed for each class 
from one pixel position mode for the pixel of Interest to another. 

[0277] The normal equation summation circuit 267 executes the above- described summation, with the totality of the 
25 pixels making up the teacher picture supplied thereto as pixels of interest, whereby a normal equation shown in the 
equation (6) Is established for each pixel position mode for each class. 

[0276] A tap coefficient decision circuit 266 solves the normal equation, generated for each class, from one pixel 
position mode to another, In the normal equation summation circuit 67, to find 256 sets of the tap coefficients, from 
class to class, to send the tap coefficients thus found to the addresses corresponding to the respective classes of a 

30 coefficient table storage unit 269. 

[0279] Depending on the number of pictures provided as pictures for learning, or on the picture contents, ft may be 
an occurrence that a class is produced In which the number of the normal equations necessary to find the tap coefficients 
cannot be realized. For such class, the tap coefficient decision circuit 66 outputs e.g., default tap coefficients. 
[0280] The coefficient table storage unit 269 memorizes 256 sets of the tap coefficients for each class supplied 

35 thereto from the tap coefficient decision circuit 268. 

[0281 ] Referring now to the flowchart of Fig. 1 2, the processing (learning processing) of a learning device 270A ol 
Fig.30 is explained. 

[0282] The decimation circuit 260 of the learning device 270A Is ted as teacher data with an HD picture, as picture 
data for learning. The decimation circuit 260 at step S130 decimates pixels of the HD picture, as teacher data, to 
*o generate an SD picture, the number of pixels in the horizontal direction and In the vertical direction of which are both 
halved. 

[0283] The blocking circuit 261 atstepS131 blocks an SD picture, as quasl-teacher data, obtained at the decimation 
circuit 260. into a pixel block of 8X8 pixels, as in the case of the JPEG encoding by the encoder 21 (Fig.2). The program 
then moves to step S132. At step S132, the DCT circuit 262 sequentially reads out the pixel blocks, formed by the 

45 blocking circuit 261 , and quantizes the so read-out blocks In accordance with the same quantization table as that used 
for JPEG encoding In the encoder 21 , to form blocks each formed by the quantized DCT coefficients (DCT blocks). 
[0284] On the other hand, the normal equation summation circuit 267 is fed with an HD picture, as teacher data. The 
normal equation summation circuit 267 blocks the HD picture into high picture quality blocks of 1 6x16 pixels. At step 
S134. those of the high picture quality blocks that have not as yet been made into the high picture quality blocks of 

so interest are nowmade Into the high picture quality blocks of Interest. At step 5134, the class tap extraction circuit 265 
extracts those quantized DCT coefficients used In classifying the high picture quality block of interest, among the pixel 
blocks formed by the blocking circuit 261 , from the DCT blocks obtained In the quantization circuit 263, to form class 
taps, which are then routed to the classification circuit 266. The classification circuit 266 at step S135 classifies the 
high picture quality blocks of Interest, using the class taps from the class tap extraction circuit 265, as explained with 

55 reference to the flowchart of Fig.29, to send the resulting class code to the normal equation summation circuit 267. 
The program then moves to step S136. 

[0285] At step S136, the normal equation summation circuit 267 sets the pixels among the pixels of the high quality 
block of interest, that are as yet not rendered pixels of Interest In the raster scanning sequence, as pixels of Interest 
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For these pixels of Interest, the prediction tap extraction circuit 264 forms the same prediction taps as those formed 
by the prediction tap extraction circuit 241 of Flg.26, by extracting necessary quantized DCT coefficients from the output 
of the quantization circuit 263. The prediction tap extraction circuit 264 sends the prediction taps forthe pixels of Interest 
as pupil data to the normal equation summation circuit 267. The program then moves to step S137. 

5 [0286J At step S137, the normal equation summation circuit 267 effects summation on the matrix A and the vector 
v of the above-mentioned equation (8), on the pixels of Interest as teacher data and prediction taps (or the DCT coef- 
ficients forming the prediction taps) as pupil data. Meanwhile, this summation Is executed from one class corresponding 
to the class code from the classification circuit 266 to another and trom one pixel position mode for the pixel of Interest 
to another. 

io [0287] The program then moves to step S1 38 where the normal equation summation circuit 267 checks whether or 
not the summation has been made for all pixels of the high picture quality block of Interest as the pixels of Interest If 
It Is verified at step S1 38 that the summation has not been made for all pixels of the high picture quality block of Interest 
as the pixels of Interest, the the program reverts to step S136 where the normal equation summation circuit 267 sets 
the pixels of the high picture quality block of interest which are not as yet made Into the pixels of Interest, as pixels ot 

13 Interest. Subsequently, a similar sequence of operations Is repeated. 

[0288] If it Is verified at step S138 that the summation has been made for all pixels of the high picture quality block 
of Interest as the pixels of Interest, the program moves to step S139 t where the normal equation summation circuit 267 
verifies whether or not the totality ot the high picture quality blocks obtained from the picture as teacher data have 
been processed as the high picture quality blocks of interest. If it Is verified atstep S139 that the processing has not 

20 been performed with the totality of the high picture quality blocks obtained trom the picture as teacher as the high 
picture quality blocks of Interest, the program reverts to step 81 34 where a high picture quality block not as yet rendered 
a high picture quality block of Interest is newly rendered a high picture quality block. Subsequently a similar sequence 
of processing operations Is repealed. 

[0289] If It Is verified at step S139 that processing has been made with the totality of the high picture quality blocks, 
25 obtained from the picture as teacher data, as the high picture quality blocks of interest, that is if a normal equation has 
been acquired from one pixel position mode to another for each class, the program moves to step S1 40 where the tap 
coefficient decision circuit 268 solves the normal equation generated from one pixel position mode for each class to 
another to find 256 sets of tap coefficients corresponding to the 256 pixel position modes of the class in question to 
send the tap coefficients thus found to the addresses corresponding to the respective classes of the tap coefficient 
so storage unit 269 for storage therein. The processing then comes to a close. 

[0290] The class-based tap coefficients, stored in the tap coefficient storage unit 269, are stored in a coefficient table 
storage unit 244 of Flg.26. 

[0291 ] So, the tap coefficients stored In the coefficient table storage unit 44 have been found by learning so that the 
prediction errors (herein the square errors) of the predicted values ofthe original pixel values, obtained on linear pre- 

6 diction calculations, will be statistically minimum. The result Is that, with the coefficient conversion circuit 232A of Fig. 
26, a JPEG encoded picture can be decoded to a picture llmltlessiy proximate to the HD picture which uses the JPEG 
encoded picture as teacher data. 

[0292] Moreover, with the coefficient conversion circuit 232A. since the processing of decoding the JPEG encoded 
picture and the processing for improving the picture quality are carried out simultaneously, a decoded picture of high 
picture quality can be efficiently produced from the JPEG encoded picture. 

[0293] Flg.32 shows another Illustrative structure of the coefficient conversion circuit 232 of Flg.23. Meanwhile, parts 
or components similar to those shown In Flg.26 are depicted by the same reference numerals That Is, a coefficient 
conversion circuit 232B shown in Fig.13 is basically constructed similarly to the circuit shown in Fig.26, except that a 
dequantlzatlon circuit 271 is provided newly. 
45 [0294] In the coefficient conversion circuit 232B, shown in Flg.32, the dequantlzatlon circuit 271 is fed with block- 
based quantized DCT coefficients, obtained on entropy decoding the encoded data In the entropy decoding circuit 231 
(Flg.23). 

[0295] Meanwhile, in the entropy decoding circuit 231 , not only the quantized DCT coefficients but also the quanti- 
zation table are obtained form the encoded data. In the coefficient conversion circuit 232B of Flg.32, this quantization 
so table is supplied from the entropy decoding circuit 231 to the dequantlzatlon circuit 271 . 

[0296] The dequantlzatlon circuit 271 dequantlzes the quantized DCT coefficients from the entropy decoding circuit 
231 In accordance with the quantization table from the entropy decoding circuit 231 to send the resulting DCT coeffk. 
cfents to the prediction tap extraction circuit 241 and to the class tap extraction circuit 242. 

[0297] Thus, In the prediction tap extraction circuit 241 and In the class tap extraction circuit 242, the prediction taps 
55 and the class taps are formed for the DCT coefficients instead of for the quantized DCT coefficients. The processing 
similar to that shown in Fig. 5 is subsequently performed on the DCT coefficients. 

[0298] Thus, since the coefficient conversion circuit 232B of Flg.32 performs processing not on the quantized DCT 
coefficients but on the DCT coefficients, It Is necessary for the tap coefficients stored in the coefficient table storage 
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unit 244 to be different from those shown in Flg.26. 

[0299] Flg.33 shows another Illustrative structure ot a learning device 270B adapted tor learning the tap coefficients 
stored In the coefficient table storage unit 244 of Flg.32. The parts or components of Flg.30 corresponding to those 
shown in Flg.30 are Indicated by the same reference numerals and the corresponding explanation Is sometimes omitted 

s for clarity. That Is, the learning device 270B shown in Flg.1 4 is basically configured sfrnllarty to the circuit shown In Fig. 
30 except tnat a dequantlzatlon circuit 281 Is newly provided at back of the quantization circuit 263. 
[0300] In the learning device 270B of Flg.33, the dequantlzatlon circuit 281 dequantizes the quantized DCT coeffi- 
cients, output by the dequantlzatlon circuit 263, similarly to the dequantlzatlon circuit 271 of Flg,32. The resulting DCT 
coefficients are routed to the prediction tap extraction circuit 264 and to the class tap extraction circuit 265. 

10 [0301 ] Thus, In the prediction tap extraction circuit 264 and In the class tap extraction circuit 265, the prediction and 
class taps are formed for the DCT coefficients instead of for the quantized DCT coefficients. Subsequently, processing 
similar to that shown In Fig. 1 1 Is performed on the DCT coefficients. 

[0302] The result Is that there are produced tap coefficients that diminish the efTect of the quantization error resulting 
from quantization and dequantlzatlon of the DCT coefficients. 

is [0303] Flg.34 shows a modified Illustrative structure of the coefficient conversion circuit 232 of Flg.23. The parts or 
components of Flg.34 corresponding to those shown In Flg.26 are Indicated by the same reference numerals and the 
corresponding explanation Is sometimes omitted for clarity. That is, the coefficient conversion circuit 232C shown in 
Flg.34 Is basically configured similarly to the circuit shown in Flg.26 except that It Is destitute of the class tap extraction 
circuit 242 and the classification circuit 243. 

20 [0304] So, the coefficient conversion circuit 232C shown In Flg.34 lacks In the concept of the class, which may be 
said to be equivalent to the number of the classes being one. Thus, only one class ofthe tap coefficients Is stored In 
the coefficient table storage unit 244 such that processing is executed using this one class of the tap coefficients. 
[0305] That Is, In the coefficient conversion circuit 232C of Flg.34, the tap coefficients memorized In the coefficient 
table storage unit 244 differ from those shown In Flg.26. 

25 [0306] Fig.35 shows an illustrative structure of the learning device 270C for executing the processing of learning the 
tap coefficients to be stored In the coefficient table storage unit 244 of Flg.34. Meanwhile, the parts or components of 
Fig .35 corresponding to those shown In Fig .30 are Indicated by the same reference numerals and the corresponding 
explanation is sometimes omitted for clarity. That Is, the coefficient conversion circuit 270C shown in Fig.35 Is basically 
configured similarly to the circuit shown in Flg.30 except that it is destitute of the class tap extraction circuit 265 or the 

30 classification circuit 268. 

[0307] Thus, in the learning device 270C of Fig.35, the above-described summation is carried out without dependency 
on the class in the normal equation summation circuit 267. The tap coefficient decision circuit 268 solves the normal 
equations generated from one pixel position mode to another to find the tap coefficients. 

[0306] Fig. 36 shows another Illustrative structure of the coefficient conversion circuit 232 of Flg.23. Meanwhile, the 
& parts or components of Fig.36 corresponding to those shown in Figs. 26 or 32 are indicated by the same reference 
numerals and the corresponding explanation Is sometimes omitted for clarity. That Is, the coefficient conversion circuit 
232D shown in Fig.36 is basically configured similarly to the circuit shown in Fig .26 except that it is destitute of the 
class tap extraction circuit 242 or the classification circuit 243 and that there Is newly provided the dequantlzatlon circuit 
271. 

40 [0309] Thus, similarly to the coefficient conversion circuit 232C of Flg.34, the coefficient conversion circuit 232D of 
Fig.36 memorizes only the class coefficients of one class and processing is carried out using these one-class tap 
coefficients. 

[031 0] Moreover, in the coefficient conversion circuit 232D of Fig.36, prediction taps are formed not on the quantized 
DCT coefficients, but on the DCT coefficients, output by the dequantlzatlon circuit 271 . as In the coefficient conversion 
43 circuit 232C of Flg.32. Subsequently, processing Is further performed on the DCT coefficients. 

[031 1 ] Thus, In the coefficient conversion circuit 232D, shown In Rg.36, the tap coefficients stored In the coefficient 
table storage unit 244 differ from those shown in Fig. 26. 

[0312] Thus, Fig. 37 shows an illustrative structure of a learning device 270D executing the learning processing of 
tap coefficients to be stored in the coefficient table storage unit 244 of Fig.36. Meanwhile, the parts or components of 

50 Fig .37 corresponding to those shown in Figs .30 or 33 are indicated by the same reference numerals and the corre- 
sponding explanation Is sometimes omitted for clarity. That is, the learning device 270D shown In Flg.37 Is basically 
configured similarly to the learning device shown In Flg.37, except that it Is destitute of the class tap extraction circuit 
265 or the classification circuit 266 and that there Is newly provided the dequantlzatlon circuit 281 . 
[0313] Thus, In the learning device 270O, shown In Flg.37, prediction taps are formed in the prediction tap extraction 

55 circuit 264 for the DCT coefficients instead of for the quantized DCT coefficients. Subsequently, processing is further 
performed on the DCT coefficients. Moreover, the normal equation summation circuit 267 executes the aforementioned 
summation without dependency on the class, such that the tap coefficient decision circuit 268 solves the normal equa- 
tions generated without dependency on the class to find the tap coefficients. 
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[0314J Although the Image produced on JPEG encoding tor compression encoding a still Image Is targeted In the 
foregoing, tt Is also possible to target e.g., a picture obtained on MPEG encoding. 

[0315] That Is, Flg.38 shows an Illustrative structure of the encoder 221 when MPEG encoding Is performed. 
[0316] In the encoder 221 , shown In Flg.36, the frames or the fields, making up a moving picture to be encoded by 
s M PEG, are sequentially sent to a motion detection circuit 291 and to a calculating unit 292. 

[0317] The motion detection circuit 291 detects the motion vector tor a frame supplied thereto, on the macro-block 
basis, to send the so detected motion vector to an entropy decoding circuit 296 and to a motion compensation circuit 
300. 

[0318] If the picture supplied to the calculating unit 292 Is an l-picture (Intra-plcture), the calculating unit 292 directly 
10 routes it on the macro-block basis to the blocking circuit 293. If the picture supplied to the calculating unit 292 is a P- 
plcture (predictive coded picture) or a B-plcture (bidl regionally predicted picture), the calculating unit 292 calculates 
a difference from the reference picture supplied from the motion compensation circuit 300 to route the difference values 
to the blocking circuit 293. 

[0319] The blocking circuit 293 blocks an output of the calculating unit 292 Into pixel blocks of 8X8 pixels which are 

is supplied to a DCT circuit 294. The DCT circuit 294 applies DCT processing to the pixel blocks from the blocking circuit 
293 to route the resulting DCT coefficients to a quantization circuit 295. The quantization circuit 295 quantizes the 
resulting block-based DCT coefficients In a preset quantization step to route the resulting quantized DCT coefficients 
to the entropy decoding circuit 296. The entropy decoding circuit 296 entropy-encodes the quantized DCT coefficients 
from the quantization circuit 295 to add the motion vector from the motion detection circuit 291 and other similar needed 

20 information to output the resulting encoded data as MPEG encoding results. 

[0320] Of the quantized DCT coefficients, output by the quantization circuit 295, the I- and P-plctures need to be 
locally decoded for use as reference pictures for subsequently encoded P- and B-Plctures. Thus, these pictures are 
routed not only to the entropy decoding circuit 296 but also to the dequantlzatlon circuit 297. 
[0321 ] The dequantlzatlon circuit 297 dequantlzes the quantized DCT coefficients from the quantization circuit 295 

25 into DCT coefficients which are routed to an inverse DCT circuit 298. The inverse DCT circuit 298 applies inverse DCT 
to the DCT coefficients from the dequantlzatlon circuit 297 to output the result to a calculating unit 299. This calculating 
unit 299 Is fed not only with the output of the Inverse DCT circuit 298 but also with a reference picture output by the 
motion compensation circuit 300. if the output of the inverse DCT circuit 298 Is a P-picture, the calculating unit 299 
adds the output to an output of the motion compensation circuit 300 to decode the original picture which is supplied to 

30 the motion compensation circuit 300. If the output ofthe Inverse DCT circuit 298 Is an l-plcture, the output Is the decoded 
l-picture, so the calculating unit 299 directly routes the picture to the motion compensation circuit 300. 
[0322] The motion compensation circuit 300 motion-compensates a locally decoded picture, supplied from the cal- 
culating unit 299, In accordance with the motion vector from the motion detection circuit 291 , to route the motion- 
compensated picture as reference picture to the calculating unit 292 and to the calculating unit 299. 

35 [0323] In the decoder 222 of Flg.23, even MPEG encoded data can be efficiently decoded to a picture of a high quality. 
[0324] That is, the encoded data Is sent to the entropy decoding circuit 231 which then entropy-decodes the encoded 
data. The quantized DCT coefficients, resultingfrom this entropy decoding, motion vector and other information are 
routed from the entropy decoding circuit 231 to the coefficient decoding circuit 232D. 

[0325] The coefficient decoding circuit 232D uses the quantized DCT coefficients Q from the entropy decoding circuit 
40 231 and the tap coefficients as found by learning to effect preset predictive calculations and motion compensation In 
accordance with the motion vector from the entropy decoding circuit 231 as necessary to decode the quantized DCT 
coefficients Into pixel values of high picture quality to route the high picture quality block comprised of the high picture 
quality pixel values to the deblocking circuit 233. 

[0326] The deblocking circuit 233 deblocks the high picture quality block obtained In the coefficient decoding circuit 
45 232D to acquire and output a high picture quality decoded picture having the numbers of horizontal and vertical pixels 
e.g., twice those of the MPEG encoded picture. 

[0327] Fig. 39 shows an illustrative structure of the coefficient decoding circuit 232 of Flg.23 in case of decoding the 
MPEG encoded data by the decoder 222. Meanwhile, the parts or components corresponding to those of Flg.36 are 
depicted by the same reference numerals and are not explained specifically. That is, the coefficient decoding circuit 
so 232E, shown In Flg.39, Is basically formed similarly to that shown In Fig.36, except that the coefficient decoding circuit 
232E Is provided with a calculating unit 314 and a motion compensation circuit 316 In rear of the sum- of products 
circuit 245. 

[0328] Thus, In the coefficient decoding circuit 232E, ehown In Flg.39, the quantized DCT coefficients aredequantlzed 
by the dequantlzatlon circuit 271 and, using the resulting DCT coefficients, prediction taps are formed by the class tap . 
55 extraction circuit 241 . The sum-of-products circuit 245 executes predictive calculations, using the tap coefficients em- 
ploying the prediction taps and the tap coefficients stored in the coefficient table storage unit 244 to output high picture 
quality data having the number of the horizontal and vertical pixels e.g., twice those of the original picture. 
[0329] The calculating unit 314 sums an output of the sum-of-products circuit 245 to an output of the motion com- 
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pensatlon unit 31 6 as necessary to decode a picture of high picture quality having the number of the horizontal and 
vertical pixels e.g., twice those of the original picture to output the resultng high quality picture to the deblocking circuit 
233 (Flg.23). 

[0330] That Is , for an l-plcture, an output of the eum-of-producte circuit 246 le a high quality picture having the number 
5 of the horizontal and vertical pixels e.g., twice those of the original picture, so that the calculating unit 314 directly 
routes an output of the sum-of-products circuit 245 to the deblocking circuit 233. 

[0331 J For a P- or B-plcture, the output of the sum-of-products circuit 245 Is a difference between the high picture 
quality picture having the number of the horizontal and vertical pixels e.g., twice those of the .original picture and the 
high quality reference picture. So, the calculating unit 314 sums thje output ofthe sum-of-products circuit 245 to the 
10 reference picture of high picture quality supplied from the motion compensation circuit 31 6 to provide a decocted picture 
of high picture quality having the numbers of the horizontal and vertical pixels e.g., twice those of the original picture 
to output the decocted picture to the deblocking circuit 233. 

[0332] On the other hand, the motion compensation circuit 316 receives I- and P-ptetures, among the high quality 
decocted pictures, output by the calculating unit 314, and applies motion compensation employing the motion vector 
w from the entropy decoding circuit 231 (Flg.23) to the high-quality decoded I- or P-plcture to produce a high quality 
reference picture which is routed to the calculating unit 314. 

[0333] Since the numbers of the horizontal and vertical pixels are e.g., twice those of the original picture, the motion 
compensation circuit 31 6 effects motion compensation In accordance with the motion vector having the sizes In the 
horizontal and vertical directions twice those of the motion vector from the entropy decoding circuit 231 . 
2o [0334] Fig. 40 shows an Illustrative structure of a learning device 270E responsible for performing the learning 
processing of the tap coefficients to be stored In the coefficient table storage unit 244. In the drawing, the parts or 
components corresponding to those shown In Flg.37 are depicted by the same reference numerals and detailed de- 
scription is omitted for simplicity. 

[0335] In the learning device 270A, shown In Flg.40, a decimation circuit 320 Is fed with] an HD picture for learning, 
2S as teacher data. Similarly to the decimation circuit 260, the decimation circuit 320 decinates the pixels of the H D picture 
as teacher data to generate quasl-teacher data which as an SD picture has the numbers ofthe horizontal and vertical 
pixels halved. The SO picture, as quasl-teacher data, is sent to a motion vector detection circuit 321 and to a calculating 
unit 322. 

[0336] The motion vector detection circuit 321 , calculating unit 322, blocking circuit 323, DCT circuit 324, quantization 

30 circuit 325, dequantlzatlon circuit 327, Inverse DCT circuit 320, calculating unit 329 or the motion compensation circuit 
330 performs the processing similar to that of the motion vector detection circuit 291 1 calculating unit 292, blocking 
circuit 293, EXIT circuit 294, quantization circuit 295. dequantlzatlon circuit 297. Inverse DCT circuit 298, calculating 
unit 299 or the motion compensation circuit 300 of Rg.3B, whereby the quantization circuit 1 25 outputs quantized DCT 
coefficients similar to those output by the quantization circuit 295 of FIg.38. 

35 [0337] The quantized DCT coefficients output by the quantization circuit 325 are sent to the dequantlzation circuit 
281 , which then dequanttzes the quantized DCT coefficients to DCT coefficients from the quantization circuit 325 Into 
DCT coefficients. These DCT coefficients then are supplied to the prediction tap extraction circuit 264. This prediction 
tap extraction circuit forms prediction taps from the DCT coefficients of the dequantlzation circuit 281 to send the so- 
formed tap coefficients as pupil data to the normal equation summation circuit 267. 

40 [0338] On the other hand, the HD picture as teacher data Is sent not only to the decimation circuit 320 but also to 
the calculating unit 332. This calculating unit subtracts an output of an Interpolation circuit 331 from the HD picture as 
teacher data as necessary to send the resulting difference to the normal equation summation circuit 267. 
[0339] That is, the interpolation circuit 331 generates a high picture quality reference picture having the numbers of 
pixels In the horizontal and vertical directions twice those ofthe reference picture ofthe SD picture output by the motion 

45 compensation circuit 330. 

[0340] If the HD picture sent to the calculating unit 332 is an l-plcture, the calculating unit sends the HD picture of 
the l-plcture directly as teacher data to the normal equation summation circuit 267. If the HD picture sent to the calcu- 
lating unit 332 Is a P- or B-plcture, the calculating unit calculates the difference between the HD picture of the P- or B- 
ptcture and the high picture quality reference picture output by the interpolation circuit 131 to yield a high quality version 

so of the difference with respect to the SD picture output by the calculating unit 322 (quasl-teacher data). This high quality 
version of the difference is sent as teacher data to the normal equation summation circuit 267. 
[0341 ] Meanwhile, the Interpolation circuit 331 Is able to increase the number of pixels by, for example, simple inter- 
polation. The Interpolation circuit 331 Is also able to increase the number of pixels by, for example, classification adap- 
tive processing. Moreover, the calculating unit 332 Is able to MPEG-encode, locally decode and motion-compensate 

55 the HD picture as teacher data for use as a reference picture. 

[0342] The normal equation summation circuit 267 executes the aforementioned summation of the output of the 
calculating unit 332 as teacher data and the prediction taps from the dequantlzatlon circuit 281 as pupil data to generate 
a normal equation. 
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[0343] The tap coefficient decision circuit 268 solves the normal equation generated by the normal equation sum- 
mation circuit 267 to find tap coefficients which are then sent to and stored In the tap coefficient storage unit 269. 
[0344J In the sum-of-prcducts circuit 245, shown In Flg.39, the tap coefficients, found as described above, are used 
to decode the MPEG-encoded data. Thus, the processing of decoding the MPEG-encoded data and the processing 
s of raising the picture quality of the decoded data again can be executed simultaneously. The result Is that, In the present 
embodiment, a decoded picture as an HD picture of a high picture quality, having double the numbers of the horizontal 
and vertical pixels, may be obtained efficiently. 

[0345] Meanwhile, the coefficient conversion circuit 232E of Flg.39 may also be designed without providing the de- 
quantlzatlon circuit 271 . In such case, It suffices to design the learning device 270E of Flg.40 without providing the 

to dequantizatlon circuit 281 . 

[0346] The coefficient conversion circuit 232E of Flg.39 may also be provided with a class tap extraction circuit 242 
and a classification circuit 243, as in the case of Flg.26. In such case, the learning device 270E of Flg.40 may be 
provided with the class tap extraction circuit 265 and a classification circuit 266, as in the case of Fig.30.. 
[0347] In the above-described case, the decoder 222 Is adapted for producing a decoded picture having the spatial 

is resolution Improved to twice that of the original picture. It Is however possible for the decoder 222 to produce a decoded 
picture having a spatial resolution raised by a factor of an optional multiple number relative to that of the original picture 
or a decoded picture having a temporal resolution Improved with respect to that of the original picture. 
[0348] That Is, It the picture to be MPEG encoded Is of a low temporal resolution, as shown In Flg;41 A, It Is possible 
for the decoder 222 to decode the original picture with a temporal resolution twice that of the original picture as shown 

20 m Flg.41 B. Moreover, if the picture of an object to be MPEG encoded Is a picture of 24-frame-per-second rate as used 
In a motion picture, as shown In Flg.42A, It Is possible for the decoder 222 to decode the encoded data, obtained on 
MPEG encoding the picture, Into a picture of 60-frame-per-aecond rate in which the temporal resolution Is 60724 tlmee 
that of the original picture, as shown In Flg.42B. In this case, the so-called 2-3 pull-down may be achieved readily. . 
[0349] If the temporal resolution Is to be Improved In the decoder 222, the prediction tap or the dass tap may be 

25 formed by two or more DCT coefficients, as shown for example in Fig. 43. 

[0350] The decoder 222 Is also able to produce a decoded picture In which not only one but also both of the spatial 
resolution and temporal resolution are Improved. 

[0351] According to the present Invention, described above, It Is possible to produce the tap coefficients found by 
learning, to use the tap coefficients and the transformed data to effect preset predictive calculations to decode the 
30 transformed data Into original data, and to produce processed data corresponding to the original data processed In a 
preset fashion. Thus, it becomes possible to decode the transformed data efficiently to process the decoded data in a 
preset fashion. 

[0352] Moreover, according to the present Invention, teacher data, as teacher, are processed In a preset fashion, 
the resulting quasMeacher data are at least orthogonal or frequency transformed to generate pupil data as pupils, and 
35 learning is executed such as to statistically minimize the prediction errors of predicted values of teacher data obtained 
on predictive calculations employing tap coefficients and pupil data to find tap coefficients. Using the tap coefficients, 
thus found, the data orthogonal or frequency transformed may be decoded eff iciently. while the decoded data may also 
be processed In a preset fashion. 

[0353] A further modification ol the present Invention Is now explained. 
40 [0354] In the embodiment, now explained, a decoder 422, shown In Fig 44, comprised of an entropy decoding circuit 
431, a coefficient conversion circuit 432 and a deblocking circuit 433, as decoder 22 shown In Fig.2, Is used to decode 
the encoded data. 

[0355] The encoded data is sent to the entropy decoding circuit 431 , which entropy decoding circuit separates the • 
encoded data into entropy encoded quantized DCT coefficients and a quantization table as the s ummational information . 
45 annexed thereto. The entropy decoding circuit 431 entropy decodes the entropy encoded quantized DCT coefficients 
to send the resulting block-based quantized DCT coefficients Q along with the quantization table, as the annexed 
Information to the coefficient conversion circuit 432. 

[0356] Using the quantization table as the annexed information, as, so to speak, an auxiliary tool, and also using tap 
coefficients, as found by learning with the quantized DCT coefficients Q, the coefficient conversion circuit 432 executes 
so preset predictive calculations to decode the block-based quantized DCT coefficients Into the original block of 8X8 
pixels. 

[0357] The deblocking circuit 433 deblocks the decoded block obtained In the coefficient conversion circuit 432 to 
produce and output the decoded picture. 

[0358] This decoder 422 performs decoding In accordance with the sequence shown in the flowchart of Fig.45 (steps 
55 S201 to S203). 

[0359] That is. the encoded data is sequentially supplied to the entropy decoding circuit 431 . At step S201, the 
entropy decoding circuit 431 entropy decodes the encoded data and sends the block-based quantized DCT coefficients 
Q to' the coefficient conversion circuit 432. "me entropy decoding circuit 431 also separates the quantization table as 
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the summational Information contained In the encoded data to send the table to the coefficient conversion circuit 432. 
The coefficient conversion circuit at step S202 executes the prediction calculations employing the quantization table 
and the tap coefficients to decode the block-based quantized DCT coefficients Q from the entropy decoding circuit 431 
into block-based pixel values which are then routed to the deblocking circuit 433. The deblocking circuit 433 at step 
s S203 deblocks the pixel values from the coefficient conversion circuit 432 (decoded block) to output the resulting 
decoded picture to terminate the processing. 

[0360] Fig. 46 shows an illustrative structure of the coefficient decoding circuit 432 of Fig 23 for decoding the quantized 
DCT coefficients into pixel values. 

[0361] in a coefficient decoding circuit 432A, shown In Flg.46, the block-based quantized DCT coefficients, output 
to by the entropy decoding circuit 431 (Fig.44), are routed to a block-based prediction tap extraction circuit 441 and to a 
class tap extraction circuit 442. The quantization table, as subsidiary information, Is routed to the classification circuit 
443. 

[0362] The prediction tap extraction circuit 441 sets the block pixel values corresponding to the block of quantized 
DCT coefficients supplied thereto, Incidentally referred to below as DCT blocks, sequentially as high picture quality 
blocks of Interest. These blocks of pixel values, not present In the present stage but virtually presupposed, are Inch 
dentally referred to below as pixel blocks. Moreover, the pixels that make up the high picture quality block of Interest 
are sequentially made into pixels of Interest In the raster scanning sequence. In summation, the prediction tap extraction 
circuit 441 extracts the quantized DCT coefficients, used for predicting the pixel values of the pixels of Interest, for use 
as prediction taps. 

20 [0363] That Is, the prediction tap extraction circuit 441 extracts, as prediction taps, the totality of the quantized DCT 
coefficients, that Is 8x8 = 64 quantized DCT coefficients, of the DCT block corresponding to the pixel block to which 
belongs the pixel of interest, as shown In FIg.6. Thus, in the present embodiment, the same prediction tap is formed 
for all pixels of a certain high picture quality block. Alternatively, the prediction tap may be formed by different quantized 
DCT coefficients from one pixel of Interest to another. 

25 [0364] The prediction taps of the respective pixels making up the pixel block, that is 64 sets of the prediction taps 
for each of the 64 pixels, obtained In the prediction tap extraction circuit 441 , are sent to a sum-of -products circuit 445. 
However, since the same prediction tap Is formed for all pixels of a pixel block, It Is In effect sufficient If a set pf the 
predication taps is provided to the eum-of-products circuit 245 for one pixel block. 

[0365] The class tap extraction circuit 442 extracts quantized DCT coefficients used for classifying the pixels ol 
30 interest into one of a plural number of classes, for use as class taps. Similarly to the prediction tap extraction circuit 
441, the class tap extraction circuit 442 extracts the totality of the 8x8 quantized DCT coefficients of the DCT block, 
corresponding to the pixel block to which belongs the pixel of Interest, shown In Flg.6. as class taps. 
[0366] It Is noted that the class tap extraction circuit 442 extracts 64 quantized DCT coefficients of a DCT block 
associated with a pixel block In question, In order to classlry the pixel blocks, from one pixel block to another, for use 
35 as class taps. 

[0367] Meanwhile, the quantized DCT coefficients , forming prediction taps or class taps, are not limited to those of 
the above-mentioned patterns. 

[0368] The class taps of the pixel block of Interest, obtained in the class tap extraction circuit 442. are supplied to 
the classification circuit 443. The classification circuit classifies the high picture quality block of Interest, based on the 
^0 class tap from the class tap extraction circuit 442, to output a class code corresponding to the so obtained class. 

[0369] As a classification method, ADRC, for example, may be used. With the method employing ADRC, the quan- 
tized DCT coefficients, foimlng a class tap, are processed with ADRC. The class of the pixel block of Interest Is deter- 
mined based on the resultant ADRC code. 

[0370] Meanwhile, in the present embodiment the classification circuit 443 extracts characteristic values of high 
45 crudaDty from the quantized DCT coefficients forming the class tap, and undertakes classification based on these 
characteristic values to diminish the number of the classes. 

[0371 ] Fig. 47 shows an Illustrative structure of the classification circuit 443 of Fig .48. 

[0372] In the classification circuit 443, shown In Fig .47, the class tap Is adapted to be supplied to a power calculating 
circuit 451 . The power calculating circuit 451 splits the quantized DCT coefficients forming the class tap into a plural 
so number of spatial frequency bands, and calculates the power values of the respective frequency bands. 

[0373] The power calculating circuit 451 splits the 8X8 quantized DCT coefficients forming the class tap Into four 
spatial frequency bands % S, , S 2 and S3 shown In Flg.6. 

[0374] The power calculating circuit 451 also finds the power values P 0 to P4 of the AC components of the quantized 
DCT coefficients for each of the spatial frequency bands S 0 to S3 to output the calculated values of the power values 
ss to the class code generating circuit 452. 

[0375] The class code generating circuit 452 compares the power values P 0 , P i , P 2 and P 3 from the power calculating 
circuit 451 to corresponding threshold values TH 0l TH,, TH 2 and TH 3 stored in a threshold value storage unit 453 to 
output class codes based on the magnitude relationships thereof. That Is, the class code generating circuit 452 com- 
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pares the power P 0 to the threshold value TH© to produce a one-bit code representing the magnitude relationship. In 
similar manner, the class code generating circuit 252 compares the power P 1 to the threshold value TH 1 to produce a 
one-bit code representing the magnitude relationship thereof. Likewise, the class code generating circuit 252 compares 
the power P 2 to the threshold value TH 2 and the power P 3 to the threshold value TH 3 to produce a one-bit code for 

s each of the power-threshold value combinations. The class code generating circuit 452 arrays the four one-bit codes 
(thus any of values from 0 to 1 5), obtained as described above, In e.g., a preset sequence, as class code representing 
the first class of the pixel block of Interest. This first class Is Incidentally referred to below as power class code. 
[0376] The class code generating circuit 452 Is ted with a quantization table as the subsidiary Information. The class 
code generating circuit 452 executes classification based o n this subsidiary Information to obtain a class code Indicating 

10 the second class of the pixel block of Interest. That Is, If, In JPEG encoding, any of the two sorts of the quantization 
tables shown In Flg.1B Is used to cany out quantization, the class code generating circuit 452 verifies which one of 
these two sorts of the quantization tables is the quantization table as the subsidiary Information, and renders a one- 
bit code representing the verified results the class code indicating the second class of the pixel block of interest. This 
one-bit code Is Incidentally referred to below as the class code of the subsidiary Information. 

» [0377] The class code generating circuit 452 appends the one-bit subsidiary Information class code, representing 
the second clasa, to the 4-blt power class code indicating the first class, to generate and output the ultimate class code 
for the pixel block of Interest. Thus, In the present embodiment, the ultimate class code Is of five bits, so that the pixel 
block of Interest Is classified In one of 2 5 (=32) classes, for example, In a class represented by one of values of 0 to 31 . 
[0378] Meanwhile, the method of classification based on the quantization table is not limited to the method described 

20 above. That Is, In the class code generating circuit 452, a plural number of quantization tables as reference patterns 
for comparison to the quantization table supplied as the subsidiary Information (the quantization table actually used In 
the JPEG encoding) may be provided, and such a quantization table, as a reference pattern, most proximate to the 
quantization table as the subsidiary Inf ormatlon , then may be detected to output a code corresponding to the detected 
results as the subsidiary Information class code. 

25 [0379] The threshold value storage unit 453 has stored therein threshold values THq to IHq for comparison to the 
power values P 0 to P 3 of the spatial frequency bands So to S3. 

[0380] In the above-described case, the DC component Xo of the quantized DCT coefficients is not used for classi- 
fication. It is however possible to use this DC component Xq to execute the classification. 

[0381 ] Reverting to Flg.46, the class codes, output by the classification circuit 243, as described above, are afforded 

30 as addresses to a coefficient table storage unit 444. 

[0382] The coefficient table storage unit 444 memorizes a coefficient table, having registered therein tap coefficients 
obtained by learning processing as will be explained subsequently, and outputs tap coefficients stored In an address 
associated with the class code, output by the classification circuit 443, to the sum-of-products circuit 245. 
[0383] In the present embodiment, In which the pixel blocks are classified, one class code Is obtained for the pixel 

3s block of interest. On the other hand, since the pixel block in the present embodiment Is made up of 8x8 = 64 pixels, 
64 sets of tap coefficients are required for decoding 64 pixels making up a pixel block of Interest. Thus, the coefficient 
table storage unit 444 memorizes 64 sets of tap coefficients for an address corresponding to one class code. 
[0384] The sum-of-products circuit 445 acquires the prediction tap output by the prediction tap extraction circuit 441 
and the tap coefficients output by the coefficient table storage unit 444 and, using the prediction taps and the tap 

40 coefficients, executes linear predictive processing (sum of products processing) shown In the equation (1), to output 
the pixel values ol the 8x B pixels of the pixel block ol interest as decoded results of the corresponding DCT block to 
the deblocking circuit 433 (Flg.44). 

[0385] In the prediction tap extraction circuit 441 , each pixel of the pixel block of interest sequentially becomes a 
pixel of Interest. The sum-of-products circuit 445 then performs processing In an operating mode associated with the 
45 position of a pixel which Is the prevailing pixel of Interest of the pixel block of Interest. This operating mode Is referred 
to as pixel position mode. 

[0386] That Is, If a number I pixel of the pixels ol the pixel block of Interest, In the raster scanning sequence, Is 
denoted p,, with the pixel p, being the prevailing pixel of Interest, the sum-of-products circuit 445 performs the processing 
of the pixel mode #1 . 

50 [0387] Specifically, the coefficient table storage unit 444 outputs 64 sets of tap coefficients for decoding each of the 
64 pixels making up a pixel block of Interest. If the set of tap coefficients for decoding the pixel pj Is denoted W, ( and 
the operating mode Is the pixel mode #1 , the sum-of-products circuit 445 performs the sum of product processing ofthe 
equation (1 ) t using the prediction tape and theeet W„ among the tap coefficients of the 64 sets, and renders the results 
of the sum of product processing the results of the decoding of the pixel pj. 

55 [0388] Referring to the flowchart of Fig.48, the processing of the coefficient conversion circuit 432A of Fig.46 is 
explained. 

[0389] The block-oased quantized DCT coefficients, output by the entropy decoding circuit 431 , are sequentially 
received by the prediction tap extraction circuit 441 and by the class tap extraction circuit 442. The prediction tap 
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extraction circuit 441 sequentially renders the pixel block corresponding to the block or the quantized DOT coefficients 
(DCT block) a pixel block ot Interest. 

[0390] At step S2 1 1 , the class tap extraction circuit 442 extracts the quantized DCT coefficients received and used 
for classifying the pixel block of Interest, to form a class tap, which then Is output to the classification circuit 443. 
s [0391] The classification circuit 443 Is fed with the class tap from the class tap extraction circuit 442, while being 
also fed with a quantization table , as the subsidiary Information, output by the entropy decoding circuit 431 . The clas- 
sification circuit 443 classifies the pixel blocks of interest, using the class tap from the class tap extraction circuit 442 
and the quantization table from the entropy decoding circuit 431, to output the resulting class code to the coefficient 
table storage unit 444. 

10 [0392] That Is, et step S212 In the flowchart of Flg.49. the power calculating circuit 451 of the classification circuit 

443 (Flg.47) splits, first at step S221 , the 6x8 quantized DCT coefficients making up the class tap Into four spatial 
frequency bands to S 3 , shown In Flg.8, to calculate the respective power values P 0 to P 3 . These power values P 0 
to P a are output from the power calculating circuit 451 to the class code generating circuit 452. 

[0393] The class code generating circuit 452 at step S222 reads out the threshold values TH 0 to TH 3 from the thresh - 
1* old value table storage unit 453 to compare the power values P 0 to P 3 from the power calculating circuit 451 to the 

threshold values TH 0 to TH 3 to generate the power class codes based on the respective magnitude relationships. 

[0394] The class code generating circuit 452 at step 5223 generates the subsidiary Information class code, using 

the quantization table, as the subsidiary Information. The class code generating circuit 452 then proceeds to step S224 

to generate an ultimate class code, from the power class code and the subsidiary information class code. The program 
so return then Is performed. 

[0395] Reverting to Hg.48, the class code, obtained at step S212 as described above, Is given from the classification 

circuit 443 to the coefficient table storage unit 444 as address. 

[0396] On receipt of the class code from the classification circuit 443 as address, the coefficient table storage unit 

444 at step S213 reads out the 64 sets of the tap coefficients stored In the address to output the tap coefficients so 
& read out to the sum-of-products circuit 445. 

[0397] The program then moves to step S2 14 where the prediction tap extraction circuit 441 extracts quantized DCT 
coefficient used for predicting the pixel values of the pixel of Interest of the pixels of the pixel block of Interest not as 
yet rendered the pixel of interest in the raster scanning sequence to form a prediction tap. This prediction tap Is sent 
from the prediction tap extraction circuft 441 to the sum-of-products circuit 445. 

30 [0398] In the present embodiment, since the same prediction taps are formed for the totality of the pixels of the pixel 
block, from one pixel block to another, it is sufficient if the processing of step S214 is performed only for the pixel initially 
made Into a pixel of Interest, while it Is unnecessary to perform the processing for the remaining 255 pixels. 
[0399] At step S215, the sum-of-products circuit 445 acquires a set of tap coefficients corresponding to the pixel 
position mode for the pixel of interest, from among the 64 set tap coefficients output at step S21 3 from the coefficient 

a* table storage unit 444 and, using the set of the tap coefficients and the prediction tap supplied from the prediction tap 
extraction circuit 441 at step S21 4, the sum-of-products circuit 445 executes the sum of product processing, shown In 
the equation (1), to acquire a value of the decoded pixel of interest. 

[0400] The program then moves to step S21 6 where the prediction tap extraction circuit 441 verifies whether or not 
the processing has been executed with the totality of the pixels of the pixel block of Interest as the pixels of Interest. If 
40 it is verified at step S11 6 that the processed has not been executed with the totality of the pixels of the pixel block ol 
Interest as the pixels of Interest, the program reverts to step S21 4 where the prediction tap extraction circuit 441 repeats 
similar processing, using a pixel of the pixel block of Interest, not as yet rendered the pixel of Interest In the raster 
scanning sequence, as pixel of interest. 

[0401 ] If It is verified at step S21 6 that the processing has been done with the totality of the pixels of the pixel block 
45 of interest as the pixels of interest, that If the decoded values of the totality of the pixels ofthe pixel block of interest, 
have been acquired, the sum-of-products circuit 445 outputs the pixel block (decoded block) made up of the decoded 
values to the deblocking circuit 433 (Flg.44) to finish the processing. 

[0402] Meanwhile, the processing conforming to the flowchart of Flg.4B Is executed repeatedly each time the pre- 
diction tap extraction circuit 441 sets a new pixel block of interest. 
so [0403] Fig.50 shows an illustrative structure of a learning device 460A executing the learning processing for tap 
coefficients stored In the coefficient table storage unit 444 of Flg.46. 

[0404] In the learning device 460A, shown In Flg.50, the blocking circuit 461 is fed with one or more pictures of picture 
data for learning as teacher data operating as teacher in learning. As in the case of JPEG encoding, the blocking circuft 
461 blocks the picture, as teacher data, into a pixel block composed of 8X8 pixels. 
ss [0405] A DCT circuit 462 sequentially reads out the pixel blocks, formed by the blocking circuit 461 , as pixel blocks 
of interest, and appDes DCT to the pixel blocks of interest to form blocks of DCT coefficients. These blocks of DCT 
coefficients are sent to the quantization circuit 463. 

[0406] The quantization circuit 463 quantizes the block of the DCT coefficients from the DCT circuit 462 In accordance 
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with the same quantization table as used for JPEG encoding, and sequentially sends the resulting blocks of the quan- 
tized DCT coefficients (DCTblocks) to the prediction tap extraction circuit 464 and tothe class tap extraction circuit 465. 
[0407] That Is, the quantization circuit 463 sets a few of routine compression ratios used In JPEG encoding and 
quantizes DCT coefficients |n accordance with the quantization table associated with each compression ratio to send 
5 the quantized DCT coefficients to the prediction tap extraction circuit 464 and to the class tap extraction circuit 465. 
The quantization circuit 463 also sends the quantization table used for quantization as the subsidiary Information to 
the classification circuit 466. Meanwhile, the same quantization table as used by the quantization circuit 463 is stored 
as a reference pattern In the class code generating circuit 452 of Flg.47. 

[0408] The prediction tap extraction circuit 464 uses a pixel of the pixels of the pixel block of Interest, which has not 
10 been rendered a pixel of Interest, as a pixel of interest, and extracts the needed quantized DCT coefficient from the 
output of the quantization circuit 463 to form the same prediction tap as that formed by the prediction tap extraction 
circuit 441 . This prediction tap Is sent from the prediction tap extraction circuit 464 to the normal equation summation 
circuit 467 as pupil data corresponding to pupils in learning. 

[0409J The class tap extraction circuit 465 extracts the needed quantized DCT coefficients from the output of the 
« quantization circuit 463, for the pixel block of Interest, to form the same class tap as that formed by the class tap 
extraction circuit 442 of FIg.46. This class tap is sent from the class tap extraction circuit 466 to the classification circuit 
466. 

[0410] Using the class tap from the class tap extraction circuit 465 and the quantization table as the subsidiary 
information from the quantization circuit 463, the classification circuit 466 classifies the pixel blocks of interest, to send 
20 the resulting class code to the normal equation summation circuit 467. 

[0411] The normal equation summation circuit 467 reads out pixel values of the pixel of Interest, as teacher data, 
from the blocking circuit 461 , and executes summation on the quantized DCT coefficients forming the prediction taps 
as pupil data from the prediction tap extraction circuit 464 and the pixel of Interest. 

[0412] The normal equation summation circuit 467 performs calculations, corresponding to the multiplication of pupil 
25 data (XjnXim) and summation (£) as respective elements in the matrix A in the above equation (8). with the use of the 

prediction taps (pupil data), for each class corresponding to the class code supplied from the classification circuit 466. 

[0413] The normal equation summation circuit 467 also executes calculations corresponding to multiplication (x in yi) 

and summation £ of pupil data and teacher data, as components of the vector v of equation (8), using the prediction 

tape (pupil data) and the pixel of Interest (teacher data), from one class corresponding to the class code supplied from 
30 the classification circuit 466 to another. 

[0414] The aforementioned summation in the normal equation summation circuit 467 is perfonned for each class 

from one pixel position mode for the pixel of Interest to another. 

[0415] The normal equation summation circuit 467 executes the above-described summation, using the totality of 
the pixels making up the teacher picture supplied thereto as pixels of Interest, whereby a normal equation shown In 

35 the equation (6) Is established for each ptxei position mode for each class. 

[0416] A tap coefficient decision circuit 468 solves the normal equation, generated for each class, from one pixel 
position mode to another, in the normal equation summation circuit 467, to find 64 sets of the tap coefficients, from 
class to class, to send the tap coeffudents thus found to the addresses corresponding to the respective classes of a 
coefficient table storage unit 469. 

<*o [0417] Depending on the number of pictures provided as pictures for learning, or on the picture contents, It may be 
an occurrence that a class Is produced In which the n umber of the normal equations necessary to find the tap coeff iciente 
cannot be produced. For such class, the tap coefficient decision circuit 468 outputs e.g., default tap coefficients. 
[041 8] The coefficient table storage unit 469 memorizes 64 sets of the tap coefficients for each class supplied thereto 
from the tap coefficient decision circuit 468. 

45 [0419] Referring now to the flowchart of Ftg.51 , the processing (learning processing) of a learning device 460A of 
Flg.50 Is explained. 

[0420] The blocking circuit 461 b fed with picture data for learning, as teacher data. The blocking circuit 461 at step 
S231 forms the picture data as teacher data Into a pixel block of 8x8 pixels as In the case the JPEG encoding. The 
program then moves to step S232, where the DCT circuit 462 sequentially reads out the pixel blocks formed by the 
so blocking circuit 461 land applies DCT to a pixel block of interest to ton n a block of DCT coefficients. The program then 
moves to step S233. 

[0421] At step S233, the quantization circuit 463 sets one of the pre-set quantization tables not as yet rendered a 
quantization table of Interest and routes the so set quantization table to the classification circuit 466. Then, program 
then moves to step S234, where the quantization circuit 463 sequentially reads out the blocks of the DCT coefficients 
55 obtained at the DCT circuit 462 to quantize the blocks in accordance with the quantization table to form blocks formed 
by quantized DCT coefficients. 

[0422] The program then moves to step S235 where the class tap extraction circuit 465 sets the pixel block, formed 
by the blocking circuit 461 but which has not been set as the pixel block of Interest, as the pixel block of Interest. 
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Moreover, the class tap extraction circuit 485 extracts the quantized DCT coefficients, used for classifying the pixel 
block of interest, from the DCT block obtained by the quantization circuit 463, to form a class tap, which then is sent 
to the classification circuit 466. As explained with reference to the flowchart of Flg.49, the classification circuit 466 at 
step S236 classifies the pixel blocks of interest, using the class tap from the class tap extraction circuit 465 and the 
s quantization table of Interest from the quantization circuit 463, and sends the resulting class code to the normal equation 
summation circuit 467. The program then moves to step S237. 

[0423] At step S237. the prediction tap extraction circuit 464 sets the pixel of the pixel block or interest which has 
not been set as the pixel of interest In the raster scanning sequence, as pixel of Interest, and extracts the needed 
quantized DCT coefficients from the output of the quantization circuit 463 for the pixel of Interest to form the same 
io prediction taps as that formed by the prediction tap extraction circuit 441. The prediction tap extraction circuit 464 
sends the prediction taps for the pixel of Interest as pupil data to the normal equation summation circuit 467. The 
program then moves to step S237. 

[0424] At step S238, the normal equation summation circuit 467 reads out the pixel of interest as teacher data from 
the blocking circuit 461 and executes the summation of the matrix A and the vector v ol the equation (8) for the quantized 
DCT coefficients forming the prediction taps as pupil data and the pixel of Interest as teacher data. Meanwhile, this 
summation is executed from one class corresponding to the class code from the classification circuit 466 to another 
and from one pixel position mode for the pixel of interest to another. 

[0425] The program then moves to step S239 where the prediction tap extraction circuit 464 checks whether or not 
the summation has been made for all pixels of the pixel block ol interest as the pixels of interest. If it is verified at step 
20 S239 that the summation has not been made for all pixels of the pixel block of Interest as the pixels of Interest, the 
program reverts to step S237 where the prediction tap extraction circuit 464 sets the pixels of the pixel block of Interest, 
which are not as yet made Into the pixels of interest, as pixels of interest. Subsequently, a similar sequence of operations 
is repeated. 

[0426] If It Is verified at step S239 that the summation has been made for all pixels of the pixel block of interest as 
2S the pixels of interest, the program moves to step S240, where the blocking circuit 461 verifies whether or not the totality 
of the pixel blocks obtained from the picture as teacher data have been processed as the pixel blocks of Interest If it 
Is verified at step S240 that the processing has not been performed wfth the totality of the pixel blocks obtained from 
a picture as teacher as the pixel blocks of Interest, the program reverts to step S235 where a pixel block not as yet 
rendered a pixel block of interest is newly rendered a pixel block. Subsequently, a similar sequence of processing 
30 operations Is repeated. 

[0427] if it is verified at step S240 that processing has been made with the totality of the pixel blocks, obtained from 
the picture as teacher data, as the pixel blocks of Interest, the program moves to step S241 where the quantization 
circuit 463 checks whether or not processing has been made with the totality of the preset quantization tables as the 
quantization table of Interest, if It Is verified at step S241 that the processing has not been made with the totality of the 
35 preset quantization tables as the quantization table of interest, the program reverts to step S233 to repeat the processing 
as from step S233 on the totality of the picture data for learning. 

[0426] If it is verified at step S241 that the processing has been made with the totality of the preset quantization 
tables as the quantization table of Interest, the program moves to step S242 where the tap coefficient decision circuit 
468 solves the normal equation generated from one pixel position mode of each class to another In the normal equation 
40 summation circuit 467 to llnd the 64 sets of the tap coefficients corresponding to the 64 pixel position modes of the 
class In question . The tap coefficients thus found are sent to and stored in the addresses corresponding to the respective 
classes of the coefficient table storage unit 469 to finish the processing. 

[0429] The class-based tap coefficients, stored in the tap coefficient storage unit 269, are stored in a coefficient table 
storage unit 244 of Flg.26. 

*3 [0430] So, the tap coefficients stored In the coefficient table storage unit 444 have been found by learning so that 
the prediction error (herein the square error) ofthe predicted values of the original pixel values, obtained on linear 
prediction calculations, will be statistically minimum. The result is that, with the coefficient conversion circuit 432A of 
Fig. 46, a JPEG encoded picture can be decoded to a picture llmltlessty proximate to the HD picture which uses the 
JPEG encoded picture as teacher data. 

50 [0431 ] Moreover, since the processing of decoding the JPEG encoded picture and the processing for improving the 
picture quality are carried out simultaneously, a decoded picture of high picture quality can be efficiently produced from 
the JPEG encoded picture. 

[0432] In the present embodiment, the classification In the classification circuits 443, 466 Is performed using not only 
the class tap but also the quantization table as the subsidiary Information. Alternatively, the classification may also be 
55 performed using only the class tap. However, if the classification is performed using the quantization table as the 
subsidiary information, as described above, what may be called a fine classification may be achieved to improve the 
picture quality of the decoded picture appreciably. 

[0433] Flg.52 shows another Illustrative structure of the coefficient conversion circuit 432 on Fig.44. Meanwhile, parts 
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or components similar to those shown In Fig. 46 are depleted by the same reference numerals That is. a coefficient 
conversion circuit 432B shown In Flg.52 Is basically constructed slmllarty to the circuit shown In Flg.46, except that a 
dequantlzatlon circuit 471 Is provided newly. 

[0434J In the coefficient conversion circuit 432B, shown in Flg.52, the dequantlzatlon circuit 471 (e fed with block- 
s based quantized DCT coefficients, obtained on entropy decoding the encoded data in the entropy decoding circuit 431 
(Flg.44). Moreover, the dequantlzatlon circuit 471 Is fed with a quantization table, as the subsidiary Information, output 
from the entropy decoding circuit 431 . 

[0435J The dequantlzatlon circuit 471 dequantlzes the quantized DOT coefficients from the entropy decoding circuit 
431 In accordance with the quantization table from the entropy decoding circuit 431, and routes the resulting DCT 
to coefficients to the prediction tap extraction circuit 441 and to the class tap extraction circuit 442. 

[0436] Thus, in the prediction tap extraction circuit 441 and In the class tap extraction circuit 442, prediction and 
class taps are formed not only for the quantized DCT coefficients but also for the DCT coefficients. Subsequently, the 
processing similar to that shown In Flg.46 Is carried out Tor the DCT coefficients. 

[0437] Thus, In the coefficient conversion circuit 432B ot Flg.52. In which processing Is carried out not only for the 
w quantized DCT coefficients but also for the DCT coefficients, the tap coefficients stored In the coefficient table storage 
unit 444 need to be different from th ose in Flg.46. 

[0436] Flg.53 shows an illustrative structure of a learning device 4605 for performing the learning processing for tap 
coefficients stored In the coefficient table storage unit 444. In Flg.53, the parts or components similar to those shown 
in Fig .50 are depicted by the same reference numerals That is, a learning device 460B, shown in Fig.53, is basically: 
20 constructed similarly to the circuit shown In Flg.50, except that a dequantlzatlon circuit 481 Is newly provided on the 
downstream side of the quantization circuit 463. 

[0439] In the learning device 460B, shown In Flg.53, the dequantlzatlon circuit 481 is fed with the quantized DCT 
coefficients output by the quantization circuit 463 and with the quantization table as the subsidiary Information. The 
dequantlzatlon circuit 481 dequantlzes the quantized DCT coefficients from the quantization circuit 463 in accordance 
25 with quantization table from similarly the quantization circuit 463 to send the resulting DCT coefficients to the prediction 
tap extraction circuit 464 and to the class tap extraction circuit 465. 

[0440] Thus, In the prediction tap extraction circuit 464 and In the class tap extraction circuit 465, the prediction tap 
and the class tap are formed not only from the quantized DCT coefficients but also from the DCT coefficients. Subse- 
quently, the processing similar to that in the case of Fig .50 is performed on the DCT coefliciehts. 
30 [0441 ] The result Is that the tap coefficients obtained suffer from the effect of the quantization error produced from 
quantization and subsequent dequantlzatlon of the DCT coefficients. 

[0442] Flg.54 shows an Illustrative structure of the coefficient conversion circuit 432 shown In Fig.44. In Flg.54, the 
parts or components similar to those shown In Flg.46 are depicted by the same reference numerals and corresponding 
description Is omitted for simplicity. That Is, the coefficient conversion circuit 432C, shown In Fig.52, Is basically con- 
3s structed similarly to the circuit shown In Flg.46, except that the coefficient conversion circuit 432 lacks in the class tap 
extraction circuit 442. ,. 

[0443] Thus, in a coefficient conversion circuit 432C. shown in Fig.54, the classification circuit 443 executes classi- 
fication based solely on the quantization table as the subsidiary Information supplied thereto, and sends the resulting 
subsidiary Information class code directly to the coefficient table storage unit 444 as the ultimate class code. 
40 [0444] In the present embodiment, the subsidiary Information class code is one bit, as described above, so that only 
tap coefficients of 2 (=2 1 ) classes of the tap coefficients are stored in the coefficient table storage unit 444, and process- 
ing is canled out using these tap coefficients. 

[0445] Thus, in the coefficient conversion circuit 432C. shown in Fig.54, the tap coefficients stored in the coefficient 
table storage unit differ from those In Flg.46. 

[0446] Flg.55 shows an Illustrative structure of a learning device 460C responsible for learning processing of the tap 
coefficients stored In the coefficient table storage unit 444 of Flg.54. In Flg.55, the parts or components similar to those 
shown in Fig, 50 aredeplcted by the same reference numerals and the corresponding description Is omitted forelmpllcity. 
That Is, the learning device 460C, shown In Flg.55, is basically constructed similarly to the circuit shown In Flg.50, 
except that the learning device 460C lacks in the class tap extraction circuit 465. 
so [0447] Thus, in the learning device 460C, shown in Flg.55, the aforementioned summation is executed In the normal 
equation summation circuit 467 from one class obtained based only on the subsidiary information to another, in the 
tap coefficient decision circuit 468, the normal equation generated by such summation is solved to find the tap coeffi- 
cient. 

[0448] Flg.56 shows an Illustrative structure of the coefficient conversion circuit 432 of Flg.44. In Fig.56, the parts or 
55 components similar to those shown in Figs.46 or 52 aredeplcted by the same reference numerals and the corresponding 
description is omitted for simplicity. That is. a coefficient conversion circuit 432D. shown in Fig.56, is basically con- 
structed similarly to the circuit shown In Flg.46. except that the coefficient conversion circuit 432D lacks In the class 
tap extraction circuit 442 and that a dequantlzatlon circuit 471 Is provided anew. 
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[0449] Thus, In the coefficient conversion circuit 432D, shown In Flg.56, as In the coefficient conversion circuit 432C 
of Flg.54, only the tap coefficients of the class obtained by classification performed based solely on the quantization 
table as the subsidiary Information are stored and processing Is carried out using these tap coefficients. 
[0460] Moreover, In the coefficient conversion circuit 432D of Flg.56, as In the coefficient conversion circuit 432C of 
5 Flg.52, the prediction taps are formed In the prediction tap extraction circuit 41 on the DCT coefficients output by the 
dequantlzation circuit 471 . Subsequently, the processing Is carried out for the DCT coefficients. 
[04511 Thus, In the coefficient conversion circuit 432D of Flg.56. the tap coefficients stored in the coefficient table 
storage unit 444 differ from those shown In Fig. 46. 

[0452] Fig. 57 shows an illustrative structure of a learning device 460D for performing learning processing of tap 
10 coefficients memorized in the coefficient table storage unit 444. Meanwhile, the parts corresponding to the case of Fig. 
50 or 53 are denoted by the same reference numerals and the corresponding explanation Is omitted for simplicity. That 
Is, the learning device 460D shown In Flg.57 Is configured basically Identically as the corresponding device shown In 
Fig.50 except that the class tap extraction circuit 465 is omitted and the dequantlzation circuit 481 is provided anew. 
[0453] Thus, with the learning device 460D, shown In Flg.57, prediction taps are formed not on the quantized DCT 
is coefficients, but on the DCT coefficients, In the prediction tap extraction circuit 464. Subsequently, processing Is carried 
out on the DCT coefficients. In the normal equation summation circuit 467, the processing Is performed for each class 
obtained on classification of the aforementioned summation based solely on the quantization table as the subsidiary 
Information. The normal equatlaon Is solved for the class-based normal equation In the tap coefficient decision circuit 
466 to find the tap coefficients. 

20 [0454] The foregoing description is made on a picture encoded In JPEG designed for compression encoding of a 
still picture. The present invention is, however, applicable to an MPEG encoded picture designed for compression 
encoding a moving picture. 

[0455] Flg.58 shows an illustrative structure of an encoder 42 of Fig.2 in case of performing M PEG encoding. 

[0456] In the encoder 421, shown In Flg.58, the frames or fields, forming a moving picture to be encoded In MPEG, 
25 are sequentially supplied to a motion detection circuit 491 and to a calculating unit 492. 

[0457] The motion detection circuit 491 detects the motion vector for the frames supplied thereto, on the macro-block 

basis, to send the detected motion vector to an entropy encoding circuit 496 and to a motion compensation circuit 500. 

[0458] If the picture sent to the motion detection circuit 491 is an l-plcture, It directly routes the picture to the blocking 

circuit 463. If the picture sent to the motion detection circuit 491 Is a P- or B-picture, it calculates the difference between 
so the picture and a reference picture supplied from the motion compensation circuit 500 to route the difference to the 

blocking circuit 493. 

[0459] The blocking circuit 493 blocks an output of the calculating unit 492 Into a pixel block of 8X8 pixels, which Is 
routed to a DCT circuit 494. The DCT circuit 494 applies DCT to the pixel block from the blocking clrcuit493 to send 
the resulting DCT coefficients to the quantization circuit 495. The quantization circuit 495 quantizes the block-based 

& DCT coefficients from the DCT circuit 493, in accordance with a preset quantization table, to send the resulting quan- 
tized DCT coefficients and the quantization table used to the entropy encoding circuit 496. The entropy encoding circuit 
496 entropy-encodes the quantized DCT coefficients from the quantization circuit 495 and appends the motion vector 
from the motion detection circuit 491 , the quantization table from the quantization circuit 495 and other needed Infor- 
mation to output the resulting encoded data as the MPEG encoding results. 

40 [0460] Of the quantized DCT coefficients, output by the quantization circuit 495, the I- and P-plctures need to be 
locally decoded for use as a reference picture for P- or B-picture encoded subsequently. So, the I- and P-plctures are 
sent to both the entropy encoding circuit 496 and the dequantlzation circuit 497. This dequantlzation circuit Is also fed 
with a quantization table used in the quantization circuit 495. 

[0461 ] The dequantlzation circuit 497 dequantlzes the quantized DCT coefficients from the quantization circuit 495 
4* in accordance with the quantization table from the quantization circuit 495 Into DCT coefficients which are sent to an 
Inverse DCT circuit 498. This Inverse DCT circuit applies Inverse DCT to the DCT coefficients from the dequantlzation 
circuit 497 to output the result to a calculating unit 499. The calculating unit 499 Is also fed not only with an output of 
the Inverse DCT circuit 498 but with the reference picture output from the motion compensation circuit 500. If the output 
from the inverse DCT circuit 498 is a P -picture, the output is summed to an output of the motion compensation circuit 
so 500 to form the decoded original picture If the output from the Inverse DCT circuit 498 is an l-plcture, the output Is a 
decoded l-plcture, so the calculating unit 499 directly sends the picture to the motion compensation circuit 500. 
[0462] The motion compensation circuit 500 applies motion compensation, In accordance with the motion vector 
from the motion detection circuit 491 , to the locally decoded picture from the calculating unit 499, to send the motion 
compensated picture as a reference picture to the calculating units 492 and 499. 
55 [0463] The encoded data, obtained on MPEG encoding, described above, can be decoded by an MPEG decoder 
510, designed as shown in Fig.59. 

[0464] in the MPEG decoder 510, shown In Flg.59, the encoded data is sent to an entropy decoding circuit 511, 
which then entropy decodes the encoded data to produce quantized DCT coefficients as well as to separate the motion 
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vector, quantization table and the other needed Information contained In the encoded data. The quantized DCT coef- 
ficients and the quantization table are routed to a dequantlzatlon circuit 51 2, while the motion vector Is sent to a motion 
compensation circuit 51 6. 

[0465] The dequantlzatlon circuit 112 dequantlzesthe quantized DCT coefficients from the entropy decoding circuit 
s 51 1 1n accordance with the quantization table supplied from the entropy decoding circuit 51 1 to form DCT coefficients 
which are sent to the Inverse DCT circuit 51 3. This Inverse DCT circuit 513 applies Inverse DCT to the DCT coefficients 
from the dequantlzatlon circuit 512 to output the result to a calculating unit 514. This calculating unit is fed not only 
with the output of the Inverse DCT circuit 51 3 but also with the already decoded I- or P-plctures output from the motion 
compensation circuit 516 and which have been motion compensated In accordance with the motlon vector from the 
to entropy decoding circuit 511 . If the output of the inverse DCT circuit 51 3 Is a P- or B-plcture, the calculating unit 51 4 
sums the output and an output ofthe motion compensation circuit 516 together to form the decoded original picture 
which Is then routed to a deblocking circuit 515. If the output of the Inverse DCT circuit 513 Is an l-plcture, the output 
Is a decoded l-plcture, so it Is directly routed to the deblocking circuit 515. 

[0466] The deblocking circuit 515 deblocks the decoded picture, supplied thereto on the pixel block basis from the 

is calculating unlt514, to produce and output a decoded picture. 

[0467] The motion compensation circuit 516 receives the I- and P-plctures, from among the decoded pictures output 
from the calculating unlt514, to apply motion compensation thereto In accordance with the motion vector from the 
entropy decoding circuit 51 1 . The motion compensation circuit 51 6 sends the motion compensated picture as reference 
picture to the calculating unit51 4. 

so [0466] With the decoder 422 of Flg.44, the MPEG encoded data can be decoded efficiently to a high quality picture, 
as described above. 

[0469] That Is, the encoded data le routed to the entropy decoding circuit 431 , which entropy decoding circuit then 
entropy decodes the encoded data to produce quantized DCT coefficients as well as to separate the motion vector, 
quantization table or the other needed Information contained In the encoded data. The quantized DCT coefficients are 
25 routed from the entropy decoding circuit 431 to the coefficient conversion circuit 432D, whilst the quantization table 
and the motion vector etc are also routed as subsidiary information from the entropy decoding circuit 431 to the coef- 
ficient conversion circuit 432D. 

[0470] Using the quantized DCT coefficients Q from the entropy decoding circuit 431, subsidiary Information, and 
the tap coefficients, as found by learning, the coefficient conversion circuit 432D performs preset predictive calculations, 
so while also performing motion compensation In accordance with the motion vector from the entropy decoding circuit 
431, as necessary, to decode the quantized DCT coefficients to the original pixel values, which are routed to the 
deblocking circuit 433. 

[0471 ] The deblocking circuit 433 deblocks the pixel block, obtained by the coefficient conversion circuit 432D, and 

which are made up of decoded pixels, to produce and output a decoded picture. 
35 [0472] Fig. 60 shows an illustrative structure of the coefficient conversion circuit 432 of Flg.44 In case of decoding 

theMPEG encoded data from the decoder 22. In Rg. 60 .the parts corresponding to the case of Fig. 56 or 59 are denoted 

by the same reference numerals and the corresponding explanation is omitted for simplicity. That is. the coefficient 

conversion circuit 432E shown In Flg.60 Is configured basically Identically as the corresponding device shown In Fig. 

56 except that the calculating unit 51 4 and the motion compensation circuit 516 shown In Flg.59 are provided down- 
40 stream of the sum-of -products circuit 45. 

[0473] Thus, In the coefficient conversion circuit 432E, shown In Flg.60, the processing similar to that shown In Fig. 

56 Is carried out In place ofthe Inverse DCT processing In the Inverse DCT circuit 513 of the MPEG decoder of Fig. 

59. Subsequently, the processing similar to that in Fig.59 is earned out to produce a decoded picture. 

[0474] Meanwhile, In the coefficient conversion circuit 432 E shown In Fig.. 60, notonly the quantization table but also 
43 the motion vector is contained in the subsidiary Information sentto the classification circuit 443. Thus, the classification 

circuit 443 Is able to perform classification based not only on the quantization table but also on the motion vector. In 

the classification based on the motion vector, it is possible to use codes representing the magnitude relationships ol 

themotlon vector and preset threshold values or codes representing the magnitude relationships of x and y components 

of the motion vector and preset threshold values, as the class codes. 
50 [0475] Flg.61 shows an illustrative structure of a learning device 460E responsible for learning processing of tap 

coefficients memorized In the coefficient table storage unit 444 ol Flg.60. in Flg.61 , the parts or components similar to 

those shown In Flg.57 are depicted by the sarrte reference numerals and the corresponding description Is omitted for 

simplicity. 

[0476] That Is, In the learning device 460E, shown In Flg.61 , a motion vector detection circuit 521 and a calculating 
55 unit 522 are fed with a picture for teaming as teacher data. The motion vector detection circuit 521 , calculating unit 
522, blocking circuit 523, DCT circuit 524. quantization circuit 525, dequantization circuit 527, inverse DCT circuit 528, 
calculating unit 529 or the motion compensation circuit 530 perform processing similar to that of the motion vector 
detection circuit 491, calculating unit 492, blocking circuit 493, DCT circuit 494, quantization circuit 495, dequantlzatlon 



39 



EP 1 282 236 A1 



circuit 497, Inverse DCT circuit 498, calculating unit 499 or the motion compensation circuit 500 on Flg.58, whereby 
the quantization circuit 525 outputs tne quantized DCT coefficients and the quantization table similar to those output 
by the quantization circuit 495 of Flg.58. 

[0477] The quantized DCT coefficients and the quantization table, output by the quantization circuit 525, are sent to 
s the dequantization circuit 481 , which then dequantlzes the quantized DCT coefficients from the quantization circuit 525 
In accordance with the quantization step from the quantization circuit 525 for conversion to DCT coefficients which 
then are supplied to the prediction tap extraction circuit 464. The prediction tap extraction circuit 464 forms prediction 
taps from the DCT coefficients from the dequantization circuit 481 to send the so formed prediction taps to the normal 
equation summation circuit 467. 
10 [0478] The classification circuit 466 performs classification based on the quantization table output from the quanti- 
zation circuit 525. If the classification circuit 433 of Fig.60 performs classification based on the quantization table and 
the motion vector, the classification circuit 466 performs classification based on the quantization table output by the 
quantization circuit 525 and on the motion vector output from the motion vector detection circuit 521 . 
[0479] The class codes, resulting from the classification by the classification circuit 466, are sent to the normal equa- 
tlon summation circuit 467, where the aforementioned summation is executed from class to class, with an output of 
the calculating unit 622 as teacher data and with the prediction taps from the dequantization circuit 481 as pupil data, 
to generate normal equations. 

[0480] The tap coefficient decision circuit 468 solves the class-based normal equations, generated by the normal 
equation summation circuit 467, to find the class-based tap coefficients, which are sent to and stored in the coefficient 
20 table storage unit 489. 

[0481] Using the class-based tap coefficients, thus found, the sum-of-products circuit 445 of Fig. 60 decodes the 
MPEG encoded data. So, the processing of decoding the MPEG encoded pictures and the processing of improving 
the picture quality can be executed simultaneously with the result that a decoded picture of high picture quality may 
be produced efficiently from the MPEG encoded picture. 
25 [0482] Meanwhile, the coefficient conversion circuit 432 E of Fig.60 can be formed without providing the dequantiza- 
tion circuit 471 . In such case, the learning device 460E on Flg.61 nay be formed without providing the dequantization 
circuit 481. 

[0483] It Is noted that the coefficient conversion circuit 432D may be designed with the class tap extraction circuit 
442. In such case, it Is sufficient ifthe learning device 460E of Fig. 61 is designed with the class tap extraction circuit 465. 
^0 [0484] In the above case, the quantization table or the motion vector Is used as the subsidiary Information. However, 
as the subsidiary information, a variety of the information not mandatory to restore the DCT coefficients may also be 
used. For example. In MPEG encoded data, picture types, macro-block types and so forth may be used, besides the 
quantization table or the motion vector, as the subsidiary information. 

[0485] According to the present Invention, as described above, the data of Interest, among the original data, Is as- 
signed to one of a plural number of classes, and a tap coefficient, among the class-based tap coefficients as found by 
learning, Is acquired which corresponds to the class of the data of Interest. Using the tap coefficient corresponding to 
the class of the data of interest, and the converted data, preset predictive calculations are executed to decode the 
converted data to original data efficiently. 

[0486] Moreover, according to the present Invention, teacher data, as teacher, are processed by at least orthogonal 
to or frequency transform, to generate pupil data as pupils, and learning is executed such as to statistically minimize the 
prediction error of predicted values of teacher data obtained on predictive calculations employing tap coefficients cor- 
responding to the class of the teacher data of Interest and puplf data to find tap coefficients. Using the tap coefficients, 
thus found, the data orthogonal or frequency transformed may be decoded efficiently. 
[0487] A further modification of the present Invention Is now explained. 
45 [0488] In the embodiment, now explained, a decoder 622, shown In Fig .62, comprised of an entropy decoding circuit 
631, a coefficient conversion circuit 632 and a deblocking circuit 633, as decoder 22 shown in FIg.2, is used to decode 
the encoded data. The encoded data Includes not only the entropy encoded quantized DCT coefficients, but also 
quantization tables. 

[0489] Using the quantized DCT coefficients Q from the entropy decoding circuit 631 , and also using tap coefficients, 
so as found by learning, as later explained, the coefficient conversion circuit 632 executes preset predictive calculations 
to decode the block-based quantized DCT coefficients Into the original block of 8X8 pixels. 

[0490] The deblocking circuit 633 deblocks the decoded block obtained In the coefficient conversion circuit 632 to 
produce and output the decoded picture. 

[0491] Referring to the flowchart of Flg.63, processing by the decoder 22 of Flg.62 Is explained. 
55 [0492] The encoded data is sequentially supplied to the entropy decoding circuit 631 . At step S3 01 , the entropy 
decoding circuit 631 entropy decodes trie encoded data to send the block-based quantized DCT coefficients Q to the 
coefficient conversion circuit 632. The coefficient conversion circuit 632 at step S302 performs predictive calculations, 
employing tap coefficients, to decode the block-based quantized DCT coefficients Q to form block-based pixel values, 
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which are then supplied to the deblocking circuit 633. The deblocking circuit at step S303 deblocks the block of the 
pixel values from the coefficient conversion circuit 632 (decoded block) to output the resulting decoded picture to 
terminate the processing. 

[0493] The coefficient conversion circuit 632 of Flg.62 uses e.g., classification adaptive processing to decode the 
s quantized DCT coefficients Into pixel values. 

[0494J Flg.64 shows an Illustrative structure of the coefficient conversion circuit 632 of Flg.62 adapted for decoding 
the quantized DCT coefficients Into pixel values. 

[04951 In a coefficient conversion circuit 632A, shown In Flg.64, the block-based quantized DCT coefficients, output 
by the entropy decoding circuit 631 (Flg.62), are sent to a prediction tap extraction circuit 641 and to a class tap 
io extraction circuit 642. 

[0496J The prediction tap extraction circuit 641 sequentially sets the block pixel values corresponding to the block 
of quantized DCT, coefficients supplied thereto, Incidentally referred to below as DCT blocks, as pixel blocks of Interest. 
These blocks of pixel values, not present In the present stage but virtually presupposed, are incidentally referred to 
below as pixel blocks. Moreover, the pixels that make up the pixel block of Interest are sequentially made into pixels 

f5 of Interestlntherasterscannlngsequence. In summation, the prediction tap extraction circuit 641 extracts the quantized 
DCT coefficients, used for predicting the pixel values of the pixels of Interest, for use as prediction taps. 
[0497] That Is, the prediction tap extraction circuit 642 of the present embodiment extracts, as prediction taps, the 
totality of the quantized DCT coefficients, that Is 6x8 = 64 quantized DCT coefficients, of the DCT block corresponding 
to the pixel block to which belongs the pixel of interest as shown in Fig.65A. Thus, in the present embodiment, the 

20 same prediction tap Is formed for all pixels of a given pixel block. Alternatively, the prediction tap may be formed by 
different quantized DCT coefficients from one pixel of Interest to another. 

[0498] Meanwhile, the quantized DCT coefficients, making up a prediction tap, are not limited to the aforementioned 
pattern. 

[0499] The prediction taps of the respective pixels making up the pixel block, that Is 64 prediction taps for each of 
25 the 64 pixels, obtained in the prediction tap extraction circuit 641 , are sent to a sum-of-products circuit 645. However, 
since the same prediction tap Is formed for all pixels of a pixel block. It Is In effect sufficient If a set of the predication 
taps Is provided to the sum-of-products circuit 645 for one pixel block. 

[0500] The class tap extraction circuit 642 extracts quantized DCT coefficients used for classifying the pixels ol 

Interest Into one of a plural number of classes, for use as class taps. 
so [0501 ] Meanwhile, In JPEG encoding, a picture Is encoded (DCTed and quantized) from one pixel block to another, 

so that pixels belonging to a given pixel block are assigned to the same class by way of classification. So. the class 

tap extraction circuit 642 forms the same class tap for the respective pixels of a given pixel block. 

[0502] That Is, in the present embodiment, the class tap extraction circuit 642 extracts, as class taps, the quantized 

DCT coefficients of a DCT block corresponding to the pixel block to which belongs the pixel of Interest and f our DCT 
3s blocks lying adjacent to the upper, lower, left and right sides of the block, totaling five DCT blocks, with the number ol 

the quantized DCT coefficients being 8x8x5 = 320. 

[0503] It is noted that assigning the respective pixels belonging to a high picture quality pixel block to the same class 
Is equivalent to classifying the pixel block. Thus, It Is sufficient If the class tap extraction circuit 242 forms not 64 sets 
of the class taps for classifying each of 64 pixels making up the pixel block of Interest, but one set of the class taps for 
^0 classifying the pixel block of Interest. To this end, the class tap extraction circuit 642 extracts quantized DCT coefficients 
of the DCT block associated with the pixel block In question and quantized DCT coefficients lying adjacentto the upper, 
lower, left and right sides thereol, In order to classify the pixel block In question, from one pixel block to another, for 
use as class taps. 

[0504] Meanwhile, the quantized DCT coefficients, forming prediction taps or class taps, are not limited to those ol 
the above-mentioned patterns. 

[0505] That Is, In JPEG encoding, DCT and quantization are executed In terms of a pixel block, made up of 8X8 
pixels, as a unit, to form a DCT block made up of 8x8 quantized DCT coefficients. It may, therefore, be contemplated 
that, In decoding pixels of a given pixel block by classification adaptive processing, only the quantized DCT coefficients 
of the DCT block associated with the pixel block be used as class taps. 

so [0506] However. It attention is directed to a given pixel block In a given picture, it Is customary that certain non- 
negllglble correlation exists between the pixels of the pixel block In question and the pixels of the neighboring pixel 
blocks. Thus, by extracting quantized DCT coefficients not only from the DCT block corresponding to a certain pixel 
block but also from the other DCT blocks and by employing them as class taps, the pixel of interest can be classified 
more appropriately, with the result that the decoded picture can be higher in picture quality than If solely the quantized 

55 DCT coefficients of the DCT block corresponding to the pixel block are used as the class tap. 

[0507] In the foregoing explanation, the quantized DCT coefficients from the DCT block corresponding to a certain 
pixel block and from the Tour DCT blocks neighboring to the upper, lower, left and right sides thereof are used as class 
taps. Alternatively, the quantized DCT coefficients, as class taps, may also be extracted from the DCT blocks lying 
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obliquely with respect to the DCT block corresponding to the pixel block In question and from the DCT blocks not lying 
adjacent to but lying In the vicinity of the DCT block corresponding to the pixel block In question. That Is, there Is no 
particular limitation to the range of the DCT blocks from which to extract the quantized DCT coefficients serving as 
class tape. 

s [0508] The class taps of the pixel block of Interest, obtained In the class tap extraction circuit 642, are sent to a 
classification circuit 643. Based on the class taps from the class tap extraction circuit 642, obtained from the class tap 
extraction circuit 642, the classification circuit 643 classifies the pixel block of interest, and outputs the class code 
corresponding to the so produced class. 

[0509] As the classification method, ADRC, for example, may be used. 
w [0510] In the method employing ADRC, the quantized DCT coefficients, forming the class tap, are ADRC processed, 
and the class of the pixel block of Interest is determined based on the so produced ADRC code. 
[051 1 ] In the classification circuit 643, the Information volume of the class tap Is compressed by ADRC processing 
or vector quantization followed by classification. 

[0512] Meanwhile, In the present embodiment, the class tap Is formed by 320 quantized DCT coefficients. Thus, If 
13 classification Is performed by 1-bit ADRC processing, the number of dass code cases Is an exorbitant figure of 2 320 . 
[0513] Thus, in the present embodiment, the classification circuit 643 extracts characteristic values of high cruciality 
from the quantized DCT coefficients forming the class tap, and undertakes classification based on these characteristic 
values to diminish the number of the classes. 

[0514] That is, Fig.66 shows an Illustrative structure of the classification circuit 643 of Fig.64. 

20 [0515] In the classification circuit 643, shown In Fig.64, the class taps are adapted to be suppiled to a power calcu- 
lating circuit 651 . The power calculating circuit 651 splits the quantized DCT coefficients forming the class taps Into a 
plural number of spatial frequency bands and calculates the power values of the respective frequency bands. 
[0516] That is, the power calculating circuit 451 splits the 8x8 quantized DCT coefficients forming the class taps 
Into four spatial frequency bands Sq, S 1( S 2 and S3 shown for example In Flg.8. 

25 [0517] If now the 8 X8 quantized DCT coefficients of a DCT block are represented with sequential integers from 0 
In the raster scanning sequence, as subscripts to a letter x, as shown In Fig. 6, the spatial frequency band So Is made 
up of four quantized DCT coefficients Xq, x 1f x*, Xg, the spatial frequency band S 1 Is made up of 12 quantized DCT 
coefficients x 2 , Xg, x 4 , x s , x 6 , x 10 , x 11t x 12 , x 13 , x 14 , x 15 , the spatial frequency band Sg is made up of 12 quantized 
DCT coefficients x, s , x 17t x^, x^, x^, x 33l x^, x 41 , x^, X49, x^, x^ and the spatial frequency band S 3 Is made up of 

30 36 quantized DCT coefficients x 18 , x 19 , x^, x^, x^, x^, x^, x^, x^, x^, *31 *34. x 3s> "as- *vr> *3* *39 *42» x 43» 

X44, X45, X48, X47, XsQ, X 51 , X52. X53, X 54 , X 5S Xgg, Xgg, Xqq, X^, X33. 

[0518] Moreover, the power calculating circuit 51 calculates power values P 0 , P 1( P 2 and P 3 of the AC components 
of the quantized DCT coefficients to output the so calculated power values P 0 , P 1( P 2 and P 3 to a class code generating 
circuit 652. 

& [0519] That is, the power calculating circuit 651 finds a square sum value x t 2 + x 8 2 + Xg 2 of the AC components x 1 , 
Xe, Xg of the four quantized DCT coefficients Xo, x 1 , Xg, Xg to output the resulting square sum to the class code generating 
circuit 652 as power P 0 . The power calculating circuit 651 also linds a square sum of the AC components of the 12 
quantized DCT coefficients for the spatial frequency band S 1p that Is the totality of the 12 quantized DCT coefficients, 
to output the resulting square sum to the class code generating circuit 652 as power P 1 . The power calculating circuit 

40 651 also finds the power values P2 and P 3 for the spatial frequency bands Sg and S 3 , as In the case of the spatial 
frequency band S, , descrbed above, to output the resulting power values to the class code generating circuit 652. 
[0520] The dass code generating circuit 652 compares the power values P 0 , P 1 , P 2 and P 3 from the power calculating 
circuit 651 to corresponding threshold values TH 0 , TH 1t TH 1 and TH 3 stored in a threshold value storage unit 653 to 
output class codes based on the magnitude relationships thereof. That Is, the class code generating circuit 652 com- 

4* pares the power P 0 to the threshold value TH© to produce a one-bit code representing the magnitude relationship. In 
similar manner, the class code generating circuit 52 compares the power P., to the threshold value TH 1( the power P 2 
to the threshold value TH, and the power P 3 to the threshold value TH 3 to produce one-bit code for each of the power- 
threshold value combinations. 

[0521 ] The dass code generating circuit 652 thus acquires four one-bit codes lor each of five DCT blocks, forming 
30 the dass taps, that is one 20-bit code. The class code generating circuit 652 outputs this 20-bit code as a class code 
representing the dass of the pixel block of interest, in this case, the pixel block of Interest is assigned to one of 2 20 
classes by way of dassification. 

[0522] The threshold value storage unit 653 memorizes the threshold values TH 0 to TH 3 to be compared to the power 
values P 0 to P 3 of the spatial frequency bands Sq to 
55 [0523] In the above case, the DC component Xq of the quantized DCT coefficients is not used for dassification. 
However, the DC component Xq may also be used for dassiflcatton. 

[0524] Reverting to Fig. 64, the class codes, output by the dassification circuit 643, as described above, are afforded 
as addresses to a coefficient table storage unit 644. 
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[0525J The coefficient table storage unit 644 memorizes a coefficient table, having registered therein tap coefficients 
obtained by learning processing, and outputs tap coefficients stored In an address associated with the class code, 
output by the classification circuit 643, to the sum-of -products circuit 645. 

[0526] In the present embodiment, since the pixel blocks are classified, one class code Is obtained for each pixel 
5 block of Interest. On the other hand, since the pixel block In the present embodiment Is made up of 8x8 = 64 pixels, 

64 sets of tap coefficients are required for decoding 64 pixels making up the pixel block of Interest Thus, the coefficient 

table storage unit 644 memorizes 64 sets of tap coefficients for an address corresponding to one class code. 

[0527] The sum-of-products circuit 645 acquires the prediction tap output by the prediction tap extraction circuit 841 

and the tap coefficients output by the coefficient table storage unit 644 and, using the prediction taps and the tap 
10 coefficients, executes linear predictive processing (sum of products processing) shown In the equation (1), to output 

the pixel values of the 8x8 pixels of the pixel block of Interest, as a decoded result of the corresponding DCT block, 

to the deblocking circuit 633 (Flg.62). 

[0528] In the prediction tap extraction circuit 641 , each pixel of the pixel block or Interest sequentially proves a pixel 
of Interest The sum-of-products circuit 645 then performs processing In an operating mode associated with the position 
f5 of a pixel which Is the current pixel of Interest of the pixel block of Interest. This operating mode Is Incidentally referred 
to below as the pixel position mode. 

[0529] That Is, If a number I pixel of the pixels of the pixel block of Interest, In the raster scanning sequence, Is 
denoted p,, with the pixel p, being the pixel of Interest, the sum-of-products circuit 645 pert orms the processing of the 
pixel mode #1 . 

20 [0530] Specifically, the coefficient table storage unit 644 outputs 64 sets of tap coefficients for decoding each of the 
64 pixels making up a pixel block of Interest. If the set of tap coefficients for decoding the pixel p, Is denoted Wj, and 
the operating mode is the pixel mode #1, the sum-of-products circuit 645 performs the sum of product processing ol 
the equation (1 ), using the prediction taps and the set W,, among the tap coefficients of the 64 sets, and renders the 
results of the sum of product processing the results of the decoding of the pixel p,. 

25 [0531 ] Referring to the flowchart of Fig.67. the processing ofthe coefficient conversion circuit 632A of Fig.64 is ex- 
plained. 

[0532] The block-based DCT coefficients, output by the entropy decoding circuit 631 ,- are sequentially received by 
the prediction tap extraction drcuit 641 and by the class tap extraction circuit 642. The prediction tap extraction circuit 
641 sequentially renders the supplied pixel block corresponding to the block of the quantized DCT coefficients (DCT 

so block), a pixel block of Interest 

[0533] At step S311 . the class tap extraction circuit 642 extracts the quantized DCT coefficients received and used 
for classifying the pixel block of Interest, that is the quantized DCT coefficients of a DCT block corresponding to the 
pixel block of Interest and four DCT blocks adjacent to upper, lower; left and right sides thereof, totaling at Ave DCT 
blocks, to form a class tap, which then Is output to the classification circuit 643. / 

35 [0534] At step 831 2, the classification circuit 643 classifies the pixel block of interest, using the class tap from the 
class tap extraction circuit 642, to output the resulting class code to the coefficient table storage unit 644. 
[0535] That is, st step S312 In the flowchart of Fig. 68, the power calculating circuit 651 of the classification circuit 

643 (Fig.66) calculates, first at step S321 , the respective power values P 0 to P 3 of the four spatial frequency bands S 0 
to S 3( shown In Flg.8, for each of the five DCT blocks forming the class taps. These power values P 0 to P 3 are output 

40 from the power calculating circuit 651 to the class code generating circuit 652. 

[0536] The class code generating circuit 652 at step S322 reads out the threshold values TH 0 to TH 3 from the thresh- 
old value storage unit 653 to compare the power values P 0 to P 3 of the five DCT blocks forming the class tap from the 
power calculating circuit 651 to the threshold values TH 0 to Th^ to generate the class codes based on the respective 
magnitude relationships before program return. 

*s [0537] Reverting to Rg.67, the class code, obtained at step S31 2 as described above, Is given from the classification 
circuit 643 to the coefficient table storage unit 644 as address. 

[0538] On receipt of the class code form the classification circuit 643 as address, the coefficient table storage unit 

644 at step S313 reads out the 64 sets of the tap coefficients stored In the address to output the tap coefficients so 
read out to the sum-of-products drcuit 645. 

so [0539] The program then moves to step S314 where the prediction tap extraction circuit 641 extracts quantized DCT 
coefficients used for predicting the pixel values of the pixel of Interest of the pixel block of Interest not as yet rendered 
the pixel of Interest In the raster scanning sequence to form a prediction tap. This prediction tap is sent from the 
prediction tap extraction circuit 641 to the sum-of-products circuit 646. 

[0540] In the present embodiment, since the same prediction tap Is formed for the totality of the pixels of the pixel 
55 block, from one pixel block to another, it is sufficient if the processing of step S314 is performed only for the pixel initially 
rendered a pixel of interest, while it is unnecessary to perform the processing for the remaining 63 pixels 
[0541] At step 5315, the sum-of-products circuit 645 acquires a set of tap coefficients corresponding to the pixel 
position mode for the pixel of Interest, from among the 64 set tap coefficients output at step S13 from the coefficient 
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table storage unit 644 and, using the set or the tap coefficients and the prediction tap supplied from the prediction tap 
extraction circuit 641 , the sum-of-products circuit 645 executes the sum of product processing, shown In the equation 
(1), to acquire a value of the decoded pixel of Interest. 

[0542] The program then moves to step S31 6 where the prediction tap extraction circuit 641 verifies whether or not 
5 the processing has been executed with the totality of the pixels of the pixel block of Interest as the pixel of Interest. If 
It Is verified at step S31 6 that the processing has not been executed with the totality of the pixels of the pixel block of 
interest as the pixels of Interest, the program reverts to step S3 1 4 where the prediction tap extraction circuit 641 repeats 
similar processing with the pixel values of the pixel of interest of the pixel block of Interest not as yet rendered the pixel 
of Interest In the raster scanning sequence as the pixel of Interest 
10 [0543] if It id verified at step S316 that the processing has been done with the totality of the pixels of the pixel block 
of interest as the pixels of interest, that is If the decoded values of the totality of the pixels of the pixel block of interest 
have been acquired, the sum-of-products circuit 645 outputs the pixel block (decoded block) made up of the decoded 
values to the deblocking circuit 633 (Flg.62) to finish the processing. 

[0544] It should be noted that the processing conforming to the flowchart of Fig. 67 is carried out repeatedly each 

1* time the prediction tap extraction circuit 641 sets a new pixel block of Interest 

[0545] In the above-described embodiment the power values P 0 to P 3 of the same spatial frequency bands S 0 to 
S 3 are calculated for each ol five DCT blocks making up the class tap, and classification is performed based on the 
powervalues. Alternatively, the classification may be performed based on the power values of different spatial frequency 
bands calculated for each of the five DOT blocks making up the ciass tap. 

so [0546] That Is, for the DCT block corresponding to the pixel block of interest, Incidentally referred to below as DCT 
block of Interest, for the DCT block adjacent to the upper side of the DCT block of Interest, for the DCT block adjacent 
to the lower side of the DCT block of Interest, for the DCT block adjacent to the left side of the DCT block of interest 
and for the DCT block adjacent to the right side of the DCT block of Interest, among the five DCT blocks forming the 
class tap, as shown shaded In Flg.69, the power Pv and Ph In the high frequency band in the vertical and horizontal 

zs directions, respectively, the power Pu in the high frequency band in the vertical direction, the power Pd in the high 
frequency band In the vertical direction, the power PI In the high frequency band in the horizontal direction and the 
power Pr In the high frequency band In the horizontal direction, respectively, are calculated. Based on these power 
valu es Pv, Ph, Pu, Pd, PI and Pr, classification may be performed In the same way as explained with reference to Rgs. 
66 and 68. 

30 [0547] In this case, the processing shown in Flg.70 is carried out in the classification cl rcu It 643 at step S31 2 of Flg.67. 
[0548] That is, at step S331 , the power calculating circuit 651 of the classification circuit 643 (Fig.66) calculates the 
powervalues Pv. Ph, Pu, Pd, PI and Pr of the respective frequency bands, as explained with reference to Flg.69, to 
output the result to the class code generating circuit 652. 

[0549] The class code generating circuit 652 at step S332 reads out a threshold value from the threshold value 

a* storage unit 653. it is assumed that threshold values THv, THh, THu, THd, THI and THr, for comparison with the power 
values Pv, Ph, Pu, Pd, PI and Pr, respectively, are stored in the threshold value storage unit 653. 
[0550] On reading out the threshold values THv, THh, THu, THd. THI and Thr, from the threshold value storage unit 
653. the class code generating circuit 652 compares these to the power values Pv, Ph, Pu, Pd, PI and Pr from the 
power calculating circuit 651 , respectively, to acquire six bits corresponding to the magnitude relationships thereof. 

40 The class code generating circuit 652 outputs a 6-bit code composed of six one-bit codes. The program then Is caused 
to return. Thus, in this case, the pixel of interest (pixel block of Interest) Is classified in any one of 64 (=2?) classes. 
[0551 ] In the foregoing, AC components of the quantized DCT coefficients, as the class taps, are used for classifi- 
cation. Alternatively, the class iflcalion may be performed using the DC component of the quantized DCT coefficients. 
[0552] That Is, the classification may be performed using the DC component Co of the DCT block of Interest and 

«* DC components of the DCT blocks adjacent to the upper, lower, left and right skies thereof, as shown In Flg.71 . 
[0553] In this case, the classification circuit 643 Is configured as shown for example In Flg.72. 
[0554] The class tap Issupplisd to a difference calculating circuit 751 . This difference calculating circuit 751 calculates 
absolute values Du, Dv, CI and Dr of the differences between the DC components Cu, Cv, CI and Cr of DCT blocks 
tying adjacent to the upper, lower, left and right sides of the DCT block of interest, forming the five DCT blocks of the 

so class tap, and the DC component Oq of the DCT block of interest, and sends the result to a class code generating 
circuit 752. That Is, the difference calculating circuit 751 calculates the following equations (9): 

D U = IC U -C 0 I 

55 D d = IC d - C 0 i 

D, = IC| - Cgl 
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D r = IC r -C 0 l (9) 

to send the calculated results to the class code generating circuit 752. 

s [0555] The class code generating ci rcu it 752 compares the results of calculations (absolute values of the differences) 
D ur D d , D| and D n as calculated by the difference calculating circuit 751 , to the corresponding threshold values THy, 
TH h , TH U , TH d , TH, and TH r stored in a threshold value storage unit 753, to output class codes based on the respective 
magnitude relationships. That is, the class code generating circuit 752 compares the absolute value of the difference 
Du to the threshold value TH U to generate a one-bit code representing Its magnitude relationship. Similarly, the class 

10 code generating circuit 752 compares the absolute vaJues of the differences D d , D| and D h to the threshold values TH d , 
TH| and TH n respectively, to produce respective one-bit codes. 

[Q556] The class code generating circuit 752 outputs 4-bit codes, obtained on arraying the four one-bit codes, ob- 
tained as described above, in a preset sequence, that is any values from 0 to 15, as class code representing the class 
of the pixel block of interest. Therefore, the pixel block of interest is assigned to one of 2* (=16) classes. 
15 [0557] The threshold value storage unit 753 memorizes the threshold values TH U , TH dl TH, and TH n for comparison 
to the absolute values of the difference values D u> D d , D| and D n respectivefy. 

[0556] If the classification circuit 543 is designed as shown in Fig.72, the processing shown in Fig.73 is carried out 
atstepS312inFig.67. 

[0559] That is, in the present case, the difference calculating circuit 751 at step S341 calculates the absolute values 
20 of the difference values D UJ D d> D, and D r , using the DC components C 0 , C U( C d , C, and C r of the five DCT blocks 
forming theclasstaps, to route the absolute values of the difference values D u , D d> D, and D r to the class code generating 
circuit 752. 

[0560] The class code generating circuit 752 at step S342 compares the threshold values THu, THd, THI and THr, 
stored in the threshold value storage unit 753, to the absolute values of the difference values H u , D d , D, and D r from 
& the difference calculating circuit 751 , to produce four one-bit codes representing the magnitude relationships. The class 
code generating circuit 752 outputs the 4-bit code, made up of the four one-bit codes. The program is then caused to 
return. 

[0561] Meanwhile, the classification may also be performed not only using only the AC or DC component of the « 
quantized DCT coefficients, but also using both the AC and DC components. Thai is, the method for classification is 
30 not limited to the above-described technique. 

[0562] It should be noted that the learning processing for learning the tap coefficients memorized in the coefficient 
table storage unit 644 of Fig.64 may be executed by the learning devices 60C and 60D shown in Figs.16 or 18. 
[0563] The coefficient conversion circuit 632 of Flg.62 may be formed as shown In Fig. 13. 

[0564] Fig. 74 shows an illustrative structure of the coefficient conversion circuit 632. In Fig. 74, the parts correspond- 
35 ing to the case of Fig.64 are denoted by the same reference numerals and the corresponding explanation is omitted 
for simplicity. That is, the coefficient conversion circuit 632B shown in Fig.74 is configured basically identically as the 
corresponding device shown in Fig 64 except that an inverse DCT circuit 701 is provided anew on the downstream 
side of the sum-of- products circuit 645. 

[0565] The Inverse DCT circuit 701 applies Inverse DCT to an output of the sum-of-products circuit 645 to decode 
40 the output into a picture, which is output. Thus, the coefficient conversion circuit 632B of Fig. 74 executes product of 
sum calculations of the quantized DCT coefficients forming the prediction taps output by the prediction tap extraction 
circuit 641 and tap coefficients stored in the coefficient table storage unit 644 to output DCT coefficients. 
[0566] Thus, in the coefficient conversion circuit 632B of Fig.74, the sum of product processing of the quantized DCT 
coefficients with the tap coefficients decodes the quantized DCT coefficients not into pixel values but into DCT coeff i- 
45 dents, which DCT coefficients are Inverse DCTed In the Inverse DCT circuit 701 andthereby decoded into pixel vaJues. 
So, the tap coefficients stored in the coefficient table storage unit 644 need to be different from those In Flg.64. 
[0567] Thus, Flg.75 shows an illustrative structure of a learning device 660B responsible for performing the learning 
processing of the tap coefficients stored in the coefficient table storage unit 644 of Fig. 74. Meanwhile, the parts or 
components corresponding to those shown in Fig.1 6 are depicted by the same reference numerals each with 6 to the 
50 left of the MSB and the corresponding explanation is omitted for clarity. That is, the learning device 660B shown in Fig. 
75 Is configured similarly to the device shown In Fig. 1 6 except that not the pixel values of the picture for learning but 
the DCT coefficients DCTed from the picture for learning, output by the DCT circuit 662, are sent as teacher data to 
the normal equation summation circuit 667. 

[0566] Thus, the normal equation summation circuit 667 in the learning device 6B0B, shown in Fig.75, executes the* 
55 above summation for the DCT coefficients output by the DCT circuit 662 as teacher data and for the quantized DCT 
coefficients forming the prediction taps output by the coefficient table storage unit 644. The tap coefficient decision 
circuit 668 solves the normal equation obtained by such summation to find tap coefficients. The result Is that, in the 
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learning device 660B of Flg.75, such tap coefficients for converting the quantized DCT coefficients Into DCT coefficients, 
In which the quantization error ascrlb able to quantization by the quantization circuit 663 Is reduced or suppressed, may 
be found. 

[0569] In the coefficient conversion circuit 632B of Fig. 74, in which the sum-of -products circuit 645 executes the sum 
s of product processing, using the aforementioned tap coefficients, an output is the quantized DCT coefficients from the 
prediction tap extraction circuit 641 converted Into DCT coefficients having the quantization error reduced. These DCT 
coefficients are inverse DCTed by the inverse DCT circuit 701 to produce a decoded picture in which the picture quality 
deterioration due to the quantization error is suppressed to a lower value. 

[0570] Fig. 76 shows a modified structure of the coefficient conversion circuit 632 shown In Fig. 62, In Flg.76, the 
10 parte corresponding to the case of Figs. 64 or 74 are denoted by the same reference numerals and the corresponding 
explanation is omitted for simplicity. The coefficient conversion circuit 632C shown In Flg.76 is configured basically 
Identically as the corresponding device shown In Rg.64 except that a dequanttzatlon circuit 671 is provided anew as 
In F1g.18 and that the Inverse DCT circuit 701 is also provided anew as in Flg.74. 

[0571 J Therefore, in the prediction tap extraction circuit 641 and In the class tap extraction circuit 642 In the coefficient 
conversion circuit 632C of Flg.76, the prediction and class taps are formed for the DCT coefficients, Instead of for the 
quantized DCT coefficients. Additionally, the sum-of-pro ducts circuit 646 in the coefficient conversion circuit 632C of 
Flg.76 executes sum of products processing, using the DCT coefficients forming the prediction taps output by the 
prediction tap extraction circuit 641 and the tap coefficients stored In the coefficient table storage unit 644 to produce 
DCT coefficients having the quantization error reduced. The so produced DCT coefficients are output to the inverse 
zo DCT circuit 701. 

[0572] Fig, 77 shows an illustrative structure of the learning device 660C responsible for performing the teaming 
processing of tap coefficients stored In the coefficient table storage unit 644 of Flg.76. In Fig.77, the parts or components 
similar to those shown in Flg.75 are depicted by the same reference numerals and the corresponding description Is 
omitted for simplicity. That Is, the learning device 660C, shown In Fig.77, Is baslcaJly constructed similarly to the drcult 
25 shown in Fig.1 9, except that the dequantization circuit 661 is provided anew as in Fig. 1 9 and that not the pixel values 
of a picture for learning but DCT coefficients from the DCT circuit 662, obtained on DCTIng the picture for learning, 
are afforded as teacher data to the normal equation summation circuit 667, as In Fig. 75. 

[0573] Thus , the learn In g device 66 0C of Fig .77 executes the at o rem entioned s ummatio n , usin g the DCT coeff Sci ents 
output by the DCT circuit 662, that Is DCT coefficients free of quantization errors, as teacher data, and also using the 
30 DCT coefficients forming the prediction taps output by a prediction tap forming circuit 664, that Is quantized and de- 
quantized DCT coefficients, as pupil data. The result is that tap coefficients for converting the quantized and dequan- 
tlzed DCT coefficients Into DCT coefficients, having reduced quantization error from quantization and dequantization. 
may be found In the learning device 660C. 

[0574] According to the present Invention, as described above, the converted data used for assigning the processing 
& data of Interest to one of a plural number of classes is extracted at least from the block other than the blocks corre- 
sponding to the processing data of Interest, for use as class tap, classification Is then executed to find the class of the 
processing data of interest, based on the class lap, and preset predictive calculations are executed, using the tap 
coefficients of the class of the processing data of interest and the converted data, to find the prediction value for 
processing data of interest, so that desired data for processing may be found efficiently from the converted data. 
to [0575] Moreover, according to the present Invention, the pupil data used for assigning the teacher data of Interest 
to one of a plural number of classes Is extracted from the block other than the block corresponding to at least the 
teacher data of Interest, for use as a class tap, based on which class tap the classification for finding the class of the 
teacher data of interest is performed. The learning is executed such as to statistically minimize the prediction errors 
of prediction values of teacher data, obtained on predictive calculations for the class-based tap coefficients and the 
*s pupil data, to find tap coefficients from class to class. Using the so found tap coefficients, desired data can be found 
efficiently from the data from orthogonal or frequency transform. 
[0576] A further modification of the present Invention is explained. 

[0577] In the modification, now explained, encoded data is decoded using a decoder 822, comprised of an entropy 
decoding circuit 631 , a coefficient conversion circuit 832 and a deblocking circuit 833, as the decoder 22 shown in Fig .2. 
so [Q578] The encoded data is supplied to the entropy decoding circuit 831 . The entropy decoding circuit 831 entropy- 
decodes the encoded data and sends the resulting block- based quantized DCT coefflclenls Q to the coefficient con- 
version circuit 832. Meanwhile, the encoded data Includes not only the entropy-encoded quantized DCT coefficients 
but also a quantization table, while the quantization table may be used as necessary for decoding the quantized DCT 
coefficients. 

55 [0579] Using the quantized DCT coefficients Q from the entropy decoding circuit 831 and the tap coefficients as 
found by teaming, the coefficient conversion circuit 832 performs preset predictive calculations to decode the block- 
based quantized DCT coefficients Into an original block of 8X8 pixels. 

[0580] The deblocking circuit 633 deblocks the decoded block obtained by the coefficient conversion circuit 832 to 
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produce and output a decoded picture. 

[0581] Referring to the flowchart of Flg.79, the processing by the decoder 822 of Flg.78 Is explained. 

[05821 The encoded data Is sequentially fed to the entropy decoding circuit 831 . At step S401 , the entropy decoding 

circuit 831 entropy decodes the encoded data to send the block-based quantized DCT coefficients to the coefficient 

5 conversion circuit 832. The coefficient conversion circuit 832 at step S402 executes predictive calculations employing 
tap coefficients to decode the block-based quantized DCT coefficients Q Into block-based pixel values which are then 
routed to the deblocking circuit 833. The deblocking circuit 833 deblocks the block of the pixel values from the coefficient 
conversion circuit 832 (decoded block) to output the resulting decoded picture to terminate the processing. 
[0583] This coefficient conversion circuit 832 of Flg.78 exploits the classification adaptive processing to decode the 

10 quantized DCT coefficients Into pixel values. 

[0584] Flg.80 shows an Illustrative structure of the coefficient conversion circuit 832 of Flg.78 used for decoding the 
quantized DCT coefficients Into pixel values. 

[0585] In thlscoefflclent conversion circuit 832A, shown in Flg.80, theblock-based quantized DCT coefficients, output 
from the entropy decoding circuit 831 (Flg.78), are sent to a prediction tap extraction circuit 841 and to a class tap 
is extraction circuit 842. 

[0586] The prediction tap extraction circuit 841 sets the block of pixel values corresponding to the block of quantized 
DCT coefficients, supplied thereto, Incidentally referred to below as DCT blocks, sequentially as pixel blocks of Interest. 
These blocks of pixel values, are not present In thepresent stage but virtually presupposed, and are Incidentally referred 
to below as pixel blocks. Moreover, the pixels that make up the pixel block of interest are sequentially made into pixels 
2v of Interest In the raster scanning sequence. In summation, the prediction tap extraction circuit 841 references the 
pattern table of a pattern table storage unit 846, for use as prediction taps. 

[0587] That Is, the pattern table storage unit 846 memorizes a pattern table, having registered therein the pattern 
Information Indicating the position relationships of the quantized DCT coefficients, extracted as prediction tap for the 
pixel of interest, with respect to the pixels of Interest. Based on the pattern Information, the prediction tap extraction 
25 circuit 841 extracts the quantized DCT coefficients to form a prediction tap for the pixel of interest. 

[0588] The prediction tap extraction circuit 841 forms prediction taps tor respective pixels, forming a pixel block of 
8X6 = 64 pixels, that Is 64 sets of prediction taps for each of the 64 pixels, as described above, to send the prediction 
taps thus formed to the sum-of -products circuit 845. 

[0589] The class tap extraction circuit 842 extracts quantized DCT coefficients used for assigning the pixel of Interest 
30 to any of a plural number of classes, for use as class taps. In JPEG encoding, In which a picture is encoded (DCTed 
and quantized) from one pixel block to another, all of the pixels belonging to a given pixel block are assigned to the 
same class by way of classification. So, the class tap extraction circuit 842 forms the same class tap for respective 
pixels of a given pixel block. That ls t the class tap extraction circuit 842 extracts all of the quantized DCT coefficients, 
that Is 8X8 = 64 quantized DCT coefficients, of a DCT block corresponding to the pixel block to which belongs the 
3s pixel of Interest, as class taps, as shown In Flg.6. It Is however possible to form the class taps from quantized DCT 
coefficients that are different from one pixel of Interest to another. 

[0590] It is noted that assigning the respective pixels belonging to a given pixel block to the same class is equivalent 
to classifying the pixel block. Thus, ft is sufficient If the class tap extraction circuit 842 forms not 64 sets of the class 
taps for classifying each of 64 pixels making up the pixel block of Interest, but one set of the class taps for classifying 
40 the pixel block of Interest. To this end, the class tap extraction circuit 842 extracts 64 quantized DCT coefficients of the 
DCT block associated with the pixel block In question, for use as class taps, In order to classify the pixel block In 
question, from one pixel block to another. 

[0591] Meanwhile, the quantized DCT coefficients, forming prediction taps or class taps, are not limited tD those of 
the above-mentioned patterns. 

45 [0592] The class tap of the pixel block or Interest, obtained In the class tap extraction circuit 842, Is sent to the 
classification circuit 843. The classification circuit 843 classifies the pixel blocks of Interest, based on the class tap 
from the class tap extraction circuit 842, to output the class code corresponding to the resulting class. 
[0593] As the classification method, ADRC, tor example, may be used.. 

[0594] In the method employing ADRC, the quantized DCT coefficients, forming the class tap, are ADRC processed, 
so and the class of the pixel block of interest Is determined based on the so produced ADRC code. 

[0595] In the classification circuit 843, preferably the Information volume of the class tap Is compressed by ADRC 
processing or vector quantization followed by classification. 

[0596] Thus, in the present embodiment, the classification circuit 843 extracts characteristic values of high cruclallty, 
from the quantized DCT coefficients, forming the class tap, and undertakes the claesif batlon based on the characteristic 
55 values to diminish the number of the classes. 

[0597] Fig.81 shows an illustrative structure of the classification circuit 843 shown in Fig.80. 

[0598] The class tap Is sent to the power calculation circuit 851 , which then splits the quantized DCT coefficients, 

forming the class tap, Intoaplural number of spatial frequency bands, to calculate the power from one frequency band 
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to another. 

[0599] That Is, the power calculating circuit 851 splits the 8X8 quantized DCT coefficients, tormlrtg the class tap, 
Into four special frequency bands S 0 , S 1p S 2 and S^, shown for example In Flg.6. The spatial frequency band So Is 
made up of four quantized DCT coefficients Xq, x,, Xg, Xg, the spatial frequency band S 1 is made up of 12 quantized 
5 DCT coefficients Xg, X3, x^, x 5 , Xg, x 7 x 10 , x u , x 12 , x 13 , x 14 , x 15 , the spatial frequency band Sg is made up of 12 quantized 
DCT coefficients x 16 , x 17 , X24, Xgg, x^. x 39 , x^, x 41 , x^, x^. x 56 , x^ and the spatial frequency band 53 Is made up ol 
36 quantized DCT coefficients x 18 , Xg©, x^, Xgt. x^, *23> *26- x 27 . x 28 , x^, Xgo, Xq A , x^ x^, x^. X3 7 . Xa 8 , x^ x^, X43, 

X44, X45, X^, X47, Xsq, X 51 , X 52 , X53, X54, X55 X^q, Xg^, X^q, Xgj, X&, X$ 3 . 

[0600] Moreover, the power calculating circuit 651 calculates power values P 0) P i( P 2 and P 3 of the AC components 
10 of the quantized DCT coefficients, for each of the special frequency bands S 0 , S t , S 2 and Sg, to output the so calculated 
power values P 0l P v P 2 and P 3 to a class code generating circuit 852. 

[0601] That Is, the power calculating circuit 851 finds a square sum value x A 2 + Xq 2 + Xg 2 of the AC components x 1( 
Xg, Xg of the four quantized DCT coefficients x©, x 1 , Xe, Xg to output the resulting square sum to the class code generating 
circuit 852 as power P 0 . The power calculating circuit 851 also finds a square sum of the AC components of the 12 
1* quantized DCT coefficients for the spatial frequency band S 1f that is the totality of the 1 2 quantized DCT coefficients, 
to output the resulting square sum to the class code generating circuit S52 as power P., . The power calculating circuit 
851 also finds the power values P 2 and P s for the spatial frequency bands ^ and S 3 , respectively, as In the case of 
the spatial frequency band S 1 described above, to output the resulting power values to the class code generating circuit 
52. 

20 [0602] The class code generating circuit 852 compares the power values P 0 , p 1t P 2 and P 3 from the power calculating 
circuit 851 to corresponding threshold values TH 0 , TH|, THj and TH 3 stored In a threshold value storage unit 853 to 
output class codes based on the magnitude relationships thereof. That is, the class code generating circuit 852 com- 
pares the power P 0 to the threshold value TH 0 to produce a one-bit code representing the magnitude relationship. In 
similar manner, the class code generating circuit 852 compares the power P 1 to the threshold value TH 1 to produce a 

25 one-bit code representing the magnitude relationship thereof . Likewise, the class code generating circuit 852 compares 
the power P 2 to the threshold value TH 2 and the power P 3 to the threshold value TH 3 , respectively, to produce a one- 
bit code for each of the power-threshold value combinations. The class code generating circuit 852 outputs the four 
one-bit codes (thus any of values from 0 to 15), obtained as described above, In e.g., a preset sequence, as class 
codes representing the classes of the pixel block of Interest Thus, in the present embodiment, the pixel blocks of 

30 interest Is classified Into 2* = 1 6 classes. 

[0603] The threshold value storage unit 853 memorizes the threshold values TH 0 to TH 3 to be compared to the power 
values P 0 to P 3 of the spatial frequency bands Sq to S3, respectively. 

[0604] In the above-described case, the DC component x© of the quantized DCT coefficients Is not used for the 
classification processing. Alternatively, this DC component Xg may also be used for classification. 
35 [0605] Reverting to Fig. 80, the class codes, output by the classification circuit 843, as described above, are afforded 
as addresses to a coefficient table storage unit 844 and to a pattern table storage unit 846. 

[0606] The coefficient table storage unit 844 memorizes a coefficient table, having registered therein tap coefficients, 
obtained by learning processing, as will be explained subsequently, and outputs tap coefficients stored In an address 
associated with the class code, output by the classification circuit 643, to the sum-of-products circuit 845. 

40 [0607] In the present embodiment, In which the pixel blocks are class Hied, one class code Is obtained for the pixel 
block of interest. On the other hand, since the pixel block In the present embodiment Is made up of 8x8 = 64 pixels, 
64 sets of tap coefficients are required for decoding 64 pixels making up a given pixel block of interest Thus, the 
coefficient table storage unit 844 memorizes 64 sets of tap coefficients for an address corresponding to one class code. 
[0608] The sum-of-products circuit 845 acquires the prediction tap output by the prediction tap extraction circuit 841 

45 and the tap coefficients output by the coefficient table storage unit 844 and, using the prediction taps and the tap 
coefficients, executes linear predictive processing (sum of products processing) shown In the equation (1), to output 
the pixel values of the 8x8 pixels of the pixel block of Interest, as decoded results of the corresponding DCT block, to 
the deblocking circuit 833 (Fig.78). 

[0609] In the prediction tap extraction circuit 841 , each pixel of the pixel block of interest sequentially becomes a 
50 pixel or interest. The sum-or-products circuit 845 then performs processing In an operating mode associated with the 
position of a pixel which Is the pixel of Interest of the pixel block of Interest This operating mode is Incidentally referred 
to below as pixel position mode. 

[0610] That is, If a number i pixel of the pixels of the pixel block of Interest, in the raster scanning sequence, is 
denoted p,, with the pixel p, being the pixel of interest, the sum-of-products circuit 845 performs the processing of the 
55 pixel position mode #1 . 

[061 1 ] Specifically, the coefficient table storage unit 844 outputs 84 sets of tap coefficients for decoding each of the 
64 pixels making up a pixel block of interest If the set of tap coefficients for decoding the pixel p, Is denoted W f , and 
the operating mode Is the pixel mode #1, the sum-of-products circuit 845 performs the sum of product processing ol 
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the equation (1 ), using the prediction taps and the set W,. among the tap coefficients of the 64 sets, and renders the 
results of the sum of product processing the results of the decoding of the pixel Pj. 

[061 2] The pattern table storage unit 846 has stored therein a pattern table, In which there Is registered the pattern 
Information obtained on learning the pattern information representing an extraction pattern of the quantized DCT In- 
s formation, as later explained. The pattern table storage unit 846 outputs the pattern Information stored In an address 
associated with the class code output by the classification circuit 843 to the prediction tap extraction circuit 841 . 
[061 3] in the pattern table storage unit 846. there are stored 64 sets of the pattern Information (the pattern Information 
for each pixel position mode) for an address associated with one class code. 

[0614J Referring to the flowchart of Flg.82, the processing ofthe coefficient conversion clrcutt 832A of Flg.80 Is ex- 
10 plained. 

[06151 The block-based DCT coefficients, output by the entropy decoding circuit 831 , are sequentially received by 
the prediction tap extraction circuit 841 and by the class tap extraction circuit 842. The prediction tap extraction circuit 
B41 sequentially renders the pixel block corresponding to the block of the quantized DCT coefficients (DCT block) a 
pixel block of Interest 

13 [0616] At step S41 1 , the class tap extraction circuit 842 extracts the quantized DCT coefficients received and usable 
for classifying the pixel block of Interest, to form class taps, which are then output to the classification circuit 243. 
[0617] At step S412, the classification circuit 843 classifies the pixel block of Interest, using the class taps from the 
class tap extraction circuit 842, to output the resulting class codes to the coefficient table storage unit 844 and to the 
pattern table storage unit 846. 

20 [0616] That Is, st step S412, In the flowchart of Flg.83, the power calculating circuit 843 of the classification circuit 
843 (Flg.81) splits, first at step S421 , the 8x8 quantized DCT coefficients making up the class tap Into four spatlai 
frsquency bands S 0 to S 3 to calculate the respective power values P 0 to P 3 . These power values P 0 to P 3 are output 
from the power calculating circuit 851 to the class code generating circuit 852. 

[0619] The class code generating circuit 852 at step S422 reads out the threshold values TH 0 to TH 3 to compare 
25 the power values P 0 to P 3 Irom the power calculating circuit 851 to the threshold values TH 0 to TH 3 to generate the 
class codes based on the respective magnitude relationships. The program then is caused to return. 
[0620] Reverting to Flg.82, the class codes, obtained at step S412 as described above, are given from the classifi- 
cation circuit 843 to the coefficient table storage unit 844 and to the pattern table storage unit 846 as addresses. 
[0621 ] On receipt ofthe class codes from the classification circuit 843 as addresses, the coefficient table storage unit 
30 844 at step S413 reads out the 64 sets of the tap coefficients stored as the addresses to output the so read out tap 
coefficients to the sum-of -products circuit B45. 

[0622] The program then moves to step S41 4 where the prediction tap extraction circuit 841 extracts quantized DCT 
coefficients used for predicting the pixel values ofthe pixels of Interest of the plxei block of Interest not as yet rendered 
the pixels of Interest In the raster scanning sequence, thereby forming prediction taps. These prediction taps are sent 

3s from the prediction tap extraction circuit 841 to the sum-of-products circuit 845. 

[0623] At step S41 5, the sum-of-products circuit 845 acquires a set of tap coefficients corresponding to the pixel 
position mode for the pixels of interest, from among the 64 set tap coefficients output at step S41 3 from the coefficient 
table storage unit 844. Then, using the set of the tap coefficients and the prediction taps supplied from the prediction 
tap extraction circuit 841 , the sum-of-products circuit 845 executes the sum of product processing, shown In the equa- 

40 tk>n (1), to acquire the values of the decoded pixels of Interest. 

[0624] The program then moves to step S416 where the prediction tap extraction circuit 841 verifies whether or not 
the processing has been executed with the totality of the pixels of the pixel block of Interest as the pixels of Interest. If 
it is verified at step S41 6 that the processing has not been executed with the totality of the pixels of the pixel block ol 
interest as the pixels of Interest, the program reverts to step S41 4 wh ere me prediction tap extraction circuit 841 rep eats 

45 similar processing with the pixel values of the pixels of Interest of the pixel block of Interest, not as yet rendered the 
pixels of Interest in the raster scanning sequence, as the pixels of Interest. 

[0625] If it is verified at step S416 that the processing has been done with the totality of the pixels of the pixel block 
of Interest as the pixels of Interest, that Is if the decoded values of the totality of the pixels of the pixel block of Interest, 
have beBn acquired, the sum-of-products circuit 845 outputs the pixel block (decoded block), made up ofthe decoded 
so values, to the deblocking circuit 833 (Fig. 78) to finish the processing. 

[0626] Meanwhile, the processing conforming to the flowchart of Flg.82 is executed repeatedly each time the pre- 
diction tap extraction circuit 841 sets a new pixel bjock of Interest. . . . 

[0627] Flg.84 shows an Illustrative structure of a learning device 660A executing the learning processing for tap 
coefficients stored In the coefficient table storage unit 644 of Flg.80. 
5ff [0628] The blocking circuit 861 is fed with one or more pictures of the picture data as teacher data operating as a 
teacher in learning. The blocking circuit 861 blocks the picture as teacher data into pixel blocks each comprised of 
8X8 pixels, as In the case of JPEG encoding. 

[0629] The DCT circuit 862 sequentially reads out the pixel blocks, formed by the blocking circuit 861 , and applies 
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DCT processing to the pixel block of Interest lo form blocks of DCT coefficients. These blocks of DCT coefficients are 
sent to the quantization circuit 863. 

[0630] The quantization drcult 863 quantizes the blocks of the DCT coefficients from the DCT circuit 862, using the 
same quantization table as that used for JPEG encoding, and sequentially sends the resulting blocks of the quantized 

5 DCT coefficients (DCT blocks) to the prediction tap extracting drcult 864 and to the class tap extraction circuit 865. 
[0631 ] With the pixels of the pixel block of Interest, not as yet rendered the pixels of Interest In the raster scanning 
sequence, as the pixels of Interest, the prediction tap extracting drcult 864 references the pattern Information, read 
out from the pattern table storage un It 870, for the pixels of Interest, by extracting the needed quantized DCT coeff Idents 
from the output of the quantization circuit 863, to form the same prediction taps as those formed by the prediction tap 

10 extraction drcult 841 of Flg.80. This class taps are sent from the prediction tap extracting drcult 864 to the normal 
equation summation drcult 867. 

[0632] The class tap extraction circuit 665 extracts quantized DCT coefficients as needed from the output of the 
quantization drcult 863, for the pixel block of Interest, to form the same dass taps as those formed by the class tap 
extraction circuit 842 of Flg.80. This class taps are sent from the dass tap extraction drcult 865 to the classification 
w circuit 866. 

[0633] Using the dass taps from the class tap extraction circuit 866, the daseificatlon circuit 866 executes the same 
processing as that performed by the dassiflcation circuit 843 of Flg.80 to classify the pixel blocks of Interest to send 
the resulting class codes to the normal equation summation drcult 867 and to the pattern table storage unit 870. 
[0634] The normal equation summation circuit 867 reads out pixel values of the pixels of interest as teacher data 
20 from the blocking circuit 861 to execute the summation on the quantized DCT coeffldents forming the prediction taps 
as pupil data from the prediction tap extracting circuit 864 and on the pixels of Interest 

[0635] That is, the normal equation summation drcult 867 executes calculations corresponding to multiplication 
(Xjf^) and summation £ of pupil data, as respective elements of the matrix A of the equation (6), using prediction taps 
(pupil data) from one dass corresponding to the dass code supplied from the classification circuit 666 to another 
25 [0636] The normal equation summation circuit 867 also executes calculations corresponding to multiplication (x ln yj) 
and summation L of prediction taps (pupil data), and the pixel of interest (teacher data), as components of the vector 
v of equation (8), using the pupil data and the teacher data, from one dass corresponding to the class code supplied 
from the daseificatlon drcult 266 to another. 

[0637] The aforementioned summation in the normal equation summation circuit 867 Is performed for each dass 
30 from one pixel position mode for the pixel of Interest to another. 

[0638] The normal equation summation circu it 867 executes the above-described summation, with the totality of the 
pixels making up the teacher picture supplied thereto as pixels of Interest whereby a normal equation shown In the 
equation (8) Is established for each pixel position mode for each class. 

[0639] A tap coefficient decision circuit 888 solves the normal equation, generated for each class, from one pixel 
3s position mode to another, In the normal equation summation drcult 867 t to find 64 sets of the tap coefficients, from 
class to dass, to send the tap coefficients thus found to the addresses corresponding to the respective classes of a 
coefficient table storage unit 869. 

[0640] Depending on the number of pictures provided as pictures for learning , or on the picture contents. It may be 
an occurrence that a dass Is produced in which the number of the normal equations necessary to find the tap coeffldents 
to cannot be produced. For such class, the tap coefficient decision circuit 868 outputs e.g., default tap coefficients. 

[0641 ] The coefficient table storage unit 869 memorizes 64 sets of the tap coeffldents for each class supplied thereto 
from the tap coefficient dedslon circuit 868. 

[0642] The pattern table storage unit 870 memorizes the same pattern table as that memorized by the pattern table 
storage unit 870. The 64 sets of the pattern information, stored in the address associated with the dass codes from 
45 the classification drcult 866, are read out and sent to the prediction tap extracting circuit 864. 

[0643] Referring now to the flowchart of Flg.85, the processing (learning processing) of a learning device 860A of 
Fig. 84 is explained. 

[0644] The blocking drcult 861 , fed with picture data for learning as teacher data, blocks the picture data as teacher 
data, at step S431 , as in the case of JPEG encoding, into a pixel block of 8x8 pixels. The program then moves to step 

50 S432 where the DCT drcult 862 sequentially reads out the pixel blocks formed by the blocking circuit 661 and applies 
DCT to the pixel block of Interest to form a block of DCT coefficients. The program then moves to step S433 where 
the quantization drcult 863 sequentially reads out the blocks of the DCT coefficients obtained In the DCT drcult 862, 
and quantizes the so read out blocks, using the same quantization table as that used In JPEG encoding to form blocks 
each made up of the quantized DCT coeffldents (DCT blocks). 

55 [0645] The program then moves to step S434 where the class tap extraction circuit 865 renders the pixel block, 
among the pixel blocks formed into one block by the blocking circuit 861 , not as yet made into the pixel block of interest 
a pixel block of Interest Moreover, the class tap extraction drcult 865 extracts the quantized DCT coefficients used for 
classifying the pixel block of interest, from the DCT blocks obtained at the quantization circuit 863, to form class taps, 
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which are then routed to the classification circuit 866. As explained In connection with the flowchart of Flg.83. the 
classification circuit 866 at step S435 classifies the pixel blocks of Interest, using the class taps from the class tap 
extraction circuit 865, to route the resulting class codes to the normal equation summation circuit 867 and to the pattern 
table storage unit 870. The program then moves to step S436. 
5 [0646] Thus, the pattern table storage unit 870 reads out the 64 sets of the pattern Information stored In an address 
associated with the class code from the classification circuit 866 to send the so read out pattern Information to the 
prediction tap extracting circuit 864.. 

[0647] With the pixel of the pixel block of Interest not as yet rendered the pixel of Interest In the raster scanning 
sequence as the pixel of Interest, the prediction tap extracting circuit 864 forms the same prediction taps as those 

10 formed by the prediction tap extraction circuit 841 of Flg.80, using the pattern Information of the 64 sets of the pattern 
Information from the pattern table storage unit 870, corresponding to the pixel position mode of the pixels of Interest, 
by extracting the quantized DCT coefficients as needed from the output of the quantization circuit 863. With the pre- 
diction taps forthe pixels of Interest as pupil data, the prediction tap extracting circuit 864 sends the pupil data to the 
normal equation summation circuit 867. The program then moves to step S437. 

is [0646] At step S437, the normal equation summation circuit 867 reads out the pixels of Interest as teacher data from 
the blocking circuit 861, and executes the aforementioned summation on the quantized DCT coefficients forming the 
prediction taps as pupil data and the pixels of interest as teacher data, in the matrix A of the eq uatlon (8) and the vector 
v. It should be noted that this summation is executed from one class corresponding to the class code from the classi- 
fication circuit 866 to another and from one pixel position mode for the pixel of interest to another. 

20 [0649] The program then moves to step S438 where the prediction tap extracting circuit 864 verifies whether or not 
the summation has been done on the totality of the pixels of the pixel block of Interest as the pixels of Interest. If It is 
found at step S438 that summation has not been done on the totality of the pixels of the pixel block of Interest as the 
pixels of interest, the program reverts to step S436 where the prediction tap extracting circuit 864 renders the pixel of 
the pixel block of Interest not as yet made Into the pixel of Interest In the raster scanning sequence a pixel of Interest 

25 anew. Subsequently, a simitar sequence of operations is repeated. 

[0650] If It Is found at step S438 that summation has been done on the totality of the pixels of the pixel block of 
Interest as the pixels of Interest, the program moves to step S439 where the blocking circuit 861 verifies whether or 
not the processing has been made with all ol the pixel blocks obtained from the picture as teacher data as the pixel 
blocks of Interest. If It is found at step S439 that processing has not been made with all of the pixel blocks obtained 

so from the picture as teacher data as the pixel blocks of Interest, the program reverts to step S434 where the pixel block 
of the pixel blocks formed by the blocking circuit 86 land which has as yet not been rendered the pixel block of interest 
is newly rendered a pixel block of Interest. Subsequently, a similar sequence of operations Is repeated. 
[0651 ] If conversely It is found at step S439 that processing has been made with all of the pixel blocks obtained from 
the picture as teacher data as the pixel blocks of interest, that Is If the normal equation Is obtained for each pixel position 

36 mode for each class, the program then moves to step S440 where the tap coefficient decision circuit 868 solves the 
normal equation generated from one pixel position mode of each class to another to find the 64 sets of the tap coeffi- 
cients associated with the 64 pixel position modes of each class to cause the tap coefficients thus found to be routed 
to and stored in the addresses associated with the respective classes of the coefficient table storage unit 869. The 
processing Is then terminated. 

40 [0652] Thus, the tap coefficients ofthe respective classes stored in the coefficient table storage unit 869 are stored 
in the coefficient table storage unit 844 of Fig. 80. 

[0653] Thus, the tap coefficients stored in the coefficient table storage unit 844 have been found by learning so that 
the prediction errors (herein the square errors) of the predicted values ofthe original pixel values, obtained on linear 
prediction calculations, will be statistically minimum. The result is that, with the coefficient conversion circuit 832A of 
Flg.80, a JPEG encoded picture can be decoded to a picture limltlessly proximate to the original picture. 
[0654] Moreover, since the processing of decoding the JPEG encoded picture and the processing for Improving the 
picture quality are carried out simultaneously, a decoded picture of high picture quality can be efficiently produced from 
the JPEG encoded picture. 

[0655] Fig. 86 shows an illustrative structure of a pattern learning device 950A responsible for the learning processing 
so of the pattern Information to be stored In the pattern table storage unit B46 of Flg.80 and In the pattern table storage 
unlt.870 of Flg.84. 

[0656] A blocking circuit 951 Is fed with one or more picture of picture data for learning. As In the JPEG encoding, 
the blocking circuit 951 forms the picture for learning Into a pixel block of 8X8 pixels, as In JPEG encoding. Meanwhile, 
the picture data for learning, supplied to the blocking circuit 951 , may be the same as or different from the picture data 
55 for learning supplied to the blocking circuit 861 of the tap coefficient learning device 860A of Fig. 84. 

[0657] The blocking circuit 951 of a DCT circuit 952 sequentially reads out pixel blocks formed by the blocking circuit 
951 to apply DCT to the pixel blocks to form blocks of DCT coefficients. These blocks of DCT coefficients are sent to 
a quantization circuit 953. 
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[0658] The quantization circuit 953 quantizes the blocks of the DCT coefficients from the DCT circuit 952 In accord- 
ance with the same quantization table as that used In JPEG encoding. The blocks of the quantized DCT coefficients 
thus produced (DCT blocks) are sequentially sent to an summation circuit 954 and to a class tap extraction circuit 955. 
[0669] With the pixel blocks, obtained In the blocking circuit 951 , sequentially as the pixel blocks of interest, and with 
the pixels of the pixel blocks of Interest not as yet rendered the pixels of Interest In the raster scanning sequence, as 
the pixels of Interest, the summation circuit 954 executes summation processing for finding correlation values (cross- 
correlation values) between the pixels of Interest and the quantized DCT coefficients output by the quantization circuit 
953. 

[0660] That Is, In the learning processing for the pattern information, all of the quantized DCT coefficients of 3x3 
DCT blocks, centered about a DCT block corresponding to a given pixel block of interest to which belongs a given 
pixel of interest, are associated with the pixel of interest, as shown In Flg.87A. This operation is carried out for all of 
the pixel blocks obtained from the picture for learning, as shown In Flg.87B, to calculate correlation values between 
respective pixels of a given pixel block and respective quantized DCT coefficients of 3x3 DCT blocks centered about 
a DCT block corresponding to the pixel block. Then, for each pixel of the pixel block, a position pattern of the quantized 
DCT coefficients exhibiting larger correlation values with respect to the pixel, as Indicated by ■, Is rendered the pattern 
Information. That Is, in Fig.87C, a position pattern of the quantized DCT coefficients exhibiting larger correlation valuee 
with a pixel of the pixel block In a third column from left and in a first row from above, Is represented by ■, this position 
pattern being the pattern Information. 

[0661 ] If a pixel in a number (x+t ) column from left and in a number (y+1 ) row from above is represented by A(x,y) 
(In the present embodiment, x t y are Integers of 0 to 7 (=8 - 1 ), and the quantized DCT coefficients of 3X3 DCT blocks, 
which are centered about the DCT block corresponding to the pixel block Including the pixel, and which He In the number 
(8+1) column from left in the number t+1 row from above, are denoted by B(s,t) (e and t in the present embodfrnent 
being integers of 0 to 23 (= 8x3 - 1)), the cross-correlation values RA0t,y)B<B.t) °* tne P^ el A(x,y) with respect to the 
quantized DCT coefficients B(s,t) In a preset position relationship with respect to the pixel A(x,y), Is expressed by the 
following equation (10): 

H Ai*W>t) = S(ACx,y)-A'(x,y))(B(s,t)-B»(s,t)) 

/{V(Z(A(x > y)-A'(x,y» 2 )V(£(B(s,t)-B 1 ( Bl t)) 2 )) (10) 

[0662] In the equation (1 0), as in the following equations (1 1) to (1 3), the summation (Z) denotes the summation for 
all pixel blocks obtained from a picture for learning, A'foy) denotes an average value of pixel values at the position s 
(x,y) of the pixel block obtained from the picture in learning and B'(s,t) denotes an average value of the quantized DCT 
coefficients lying at the position (s,t) of 3X3 DCT blocks corresponding to a pixel block obtained from a picture for 
learning. 

[0663] Thus, if the total number of the pixel blocks obtained from a picture for learning is denoted N, an average 
value A' (x,y) and an average value B'(s,t) may be represented by the following equation (11): 

A'<x,y) = (ZA(x,y))/N 

B'(s,t) = (LB<s,t)VN (11). 
[0664] Substituting the equation (11) into the equation (10), the following equation (1 2): 

**roflt4> = NX(A(x,y)-B(s,Q>.(XA(x 1 y))(XB(s > t)) 

/(V(N2A(x,y) 2 -(SA(x,y)) 2 )V(N£B<s,t) 2 -(LB(s,t)) 2 )) (12) 

is derived. 

[0665] For finding the correlation value R A (x,y)B<»,Q from the equation (12), the summation calculations of the following 
five terms: 



SA(x,y) 
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summation calculations oT the following five terms: 

£A(x,y) 

5 ' £B(s t t> 

ZA(x,y) 2 

w SB(8,t) 2 

2(A(x,y)B(s,t)) (13) 

need to be performed. So, the summation circuit 954 sums these five terms together. 

13 [0666] Here, the class is not taken into consideration, for simplifying the explanation. In a pattern learning device 
950A of Fig.86, the summation circuit 954 undertakes the summation calculations for each of the class codes supplied 
from a classification circuit 956. Thus, in the above case, summation (£) denotes summation for all of the pixel blocks 
obtained from thepicture for learning. However, if the class is taken into consideration, the summation (£) of the equation 
(13) denotes the summation of the pixel blocks obtained from the picture for learning and which belong to a given one 

2o of the classes. 

[0667] Reverting to Fig.86, when the result of the summation calculations of the equation (13) for calculating the 
correlation values of pixels at respective positions of a pixel block and the quantized DCT coefficients at respective 
positions of 3X3 DCT blocks centered about the DCT block corresponding to the pixel block is obtained for each class,, 
the summation circuit 954 outputs the result of the summation calculations to a con-elation coefficient calculating circuit 
25 957. 

[0668] The class tap extraction circuit 955 forms the same class taps as those formed by the class tap extraction 
circuit B42 of Fig.BO by extracting the needed quantized DCT coefficients from the output of the quantization circuit 
953. These class taps are supplied from the class tap extraction circuit 955 to the classification circuit 956. 
[0669] The classification circuit 958 performs the same processing as that of the classification circuit 843 of Fig.80, 
30 using the class taps from the class tap extraction circuit 955, to classify the p ixel blocks of interest to send the resulting 
class codes to the summation circuit 954. 

[0670] Using the output of the summation circuit 954, the correlation coefficient calculating circuit 957 calculates the 
correlation values between the pixels at the respective positions of the pixel block and the quantized DCT coefficients 
at the respective positions of 3x3 DCT blocks centered about the DCT block corresponding to the pixel block, from 
3s class to class, in accordance with the equation (12), and sends the resulting correlation value to a pattern selection 
circuit 958. 

[0671 ] Based on the con-elation values from the correlation coefficient calculating circuit 957, the pattern selection 
circuit 958 recognizes, from class to class, the positions of the DCT coefficients showing larger correlation values with 
respect to the respective 8x8 pixels at the respective positions of the pixel block. That is, the pattern selection circuit 

40 958 recognizes, from class to class, the positions of the DCT coefficients having absolute correlative values with respect 
to the 8x8 pixels at the respective positions of the pixel blocks larger than a preset threshold value. Alternatively, the 
pattern selection circuit 958 recognizes, from class to class, the positions of the DCT coefficients having absolute 
correlative values with respect to the 6X8 pixels at the respective positions of the pixel blocks higher than a preset 
rank value. The pattern selection circuit 958 sends position patterns of 64 sets of the DCT coefficients, for each pixel 

45 position mode with respect to each of the 8x8 pixels, recognized from class to class, to a pattern table storage unit 
959* as the pattern information. 

[0672] If the pattern selection circuit 958 is designed to recognize the positions of the DCT coefficients having ab- 
solute correlative values with respect to pixels at the respective positions of the pixel block higher than a preset rank 
value, the number of the positions of the DCT coefficients thus recognized is of a fixed value corresponding to the 
so preset rank value. On the other hand, if the positions of the DCT coefficients having absolute correlative values with 
respect to pixels at the respective positions of the pixel block not lower than a preset threshold value, the number of 

the positions of the DCT coefficients is of a variable value. 

[0673] The pattern table storage unit 959 memorizes the pattern Information output by the pattern selection circuit . 
958. 

55 [0674] Referring to the flowchart of Fig.BB, the processing by the pattern learning device 950A of Fig.86 (learning 
processing) is explained. 

[0675] The blocking circuit 951 is fed with picture data for learning. At step S451 , the blocking circuit 861 forms the 
picture data for learning into a pixel block of 8X8 pixels, as in the case of the JPEG encoding. The program then moves 
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to step S452, where the DCT circuit 952 sequentially reads out the pixel block formed by the blocking circuit 951 and 
applies DCT to the pixel block to form a block of DCT coefficients. The program then moves to step S453, where the 
quantization circuit 953 sequentially reads out the biocks of DCT coefficients acquired at the DCT circuit 952 and 
quantizes the DCT coefficients in accordance with the same quantization table as that used in JPEG encoding to form 

5 a block of the quantized DCT coefficients (DCT block). 

[0676] The program then moves to step S454 where the summation circuit 954 renders the pixel block not as yet 
made into the pixel block of interest a pixel block of interest. At step S454. the class tap extraction circuit 965 extracts 
the quantized DCT coefficients, used for classifying the pixel blocks of interest, from the DCT block obtained at the 
quantization circuit 863 to form a class tap which is supplied to the classification circuit 956. In a manner explained 

10 with reference to the flowchart of Fig.83, the classification circuit 956 classifies the pixel blocks of interest, using the 
class tap from the class tap extraction circuit 955, to send the resulting class codes to the summation circuit 954. The 
program then moves to step S456. 

[0677J At step S456, thesummatlon drcuit954 performs summation processing of the equation (13), using the picture 
for learning blocked by the blocking circuit 951 and the quantized DCT coefficients output by the quantization circuit 
953, with the pixels of the pixel block of Interest not as yet made into the pixel of interest, as pixels of Interest, form 
one position of such pixel of Interest (one pixel position mode) to another, and from one class code supplied from the 
classification circuit 956 to another. The program then moves to step S457. 

[0678] At step S457, the summation circuit 954 verifies whether or not the summation calculations have been exe- 
cuted with all of pixels of the pixel blocks of interest as pixels of interest. If it Is found at step S457 that summation has 
20 not been done on the totality of the pixels of the plxei blocks of Interest as the pixels of Interest, the program moves to 
step S456 where the summation circuit 954 renders the pixels of the pixel block of Interest not as yet rendered the 
pixel of Interest in the raster sequence a new pixel of Interest. Subsequently, a similar sequence of operations is re- 
peated. 

[0679] If conversely It Is found at step S457 that processing has been made with all of the pixel blocks obtained from 

25 the picture as teacher data as the pixel blocks of interest, the program moves to step S458 where the summation circuit 
954 vertries whether or not processing has been done with all pixels of the pixel blocks of Interest obtained from the 
picture for learning as the pixel blocks of Interest. If It Is found at step S458 that processing has not been done with ali 
pixels of the pixel block of Interest obtained from the picture for learning as the pixel blocks of Interest, the program 
reverts to step S454 where a pixel block formed by the blocking circuit 951 and which has not been made Into the pixel 

so block of Interest is made Into a new pixel block ot Interest. Subsequently, a similar sequence of operations is repeated. 
[0680] If conversely it is found at step S458 that processing has been made with all of the pixel blocks obtained from 
the picture for learning as pixel blocks of Interest, the program moves to step S459 where the correlation coefficient 
calculating circuit 957 calculates the correlation values between the pixels at the respective positions of a given pixel 
block and quantized DCT coefficients at respective positions of 3X3 DCT blocks centered about the DCT block cor- 

35 responding to the pixel block to send the calculated results to the pattern selection circuit 958. 

[0681 ] Based on the correiatf on values from the correlation coefficient calculating circuit 957, the pattern selection 
circuit 958 at step S460 recognizes, from class to class, the positions of the DCT coefficients showing position rela- 
tionships of larger correlation values with respect to the 8X8 pixels at the respective positions ofthe pixel block. The 
pattern selection circuit 958 routes position patterns of 64 sets ot DCT coefficients for the respective ones of the 8X8 

to pixels at the respective positions of the pixel block, as recognized from class to class, as pattern Information, to the 
pattern table storage unit 969, for storage therein. The processing then comes to a close. 

[0682] As described above, the 64 sets of the class-based pattern Information, stored In the pattern table storage 
unft 959, are stored in the pattern table storage unit 846 of Flg.BO and in the pattern table storage unit B70 of Rg.84. 
[0683] Thus, in the coefficient conversion circuit 832A, shown in Flg.BO, the quantized DCT coefficients showing 
4* larger correlation values with respect to the pixels of Interest are extracted as prediction taps, and the quantized DCT 
coefficients are decoded to the original plxei values, with the use ol these prediction taps. Thus, the decoded picture 
may be Improved in picture quality as compared to the case of random extraction of the quantized DCT coefficients 
used as prediction taps. 

[0684] In JPEG encoding, DCT and quantization are executed In terms of a pixel block of 8X6 pixels as a unit, so 
50 that a DCT block Is formed which Is comprised of Bx8 quantized DCT coefficients. It may, therefore, be contemplated 
that, In decoding pixels of a given pixel block by classification adaptive processing, only the quantized DCT coefficients 
of the DCT block associated with the pixel block be used as class taps. 

[0685] However, if attention Is directed to a given pixel block In a given picture, It Is customary that certain non- 
negligible correlation exists between the pixels ofthe pixel block In question and the pixels of the neighboring pixel 
55 blocks. Thus, by extracting quantized DCT coefficients, showing position relationships of higher correlation with respect 
to the pixels of interest, not only from the 3X3 DCT blocks centered about the DCT block associated with the given 
pixel block, that Is DCT blocks associated with the DCT block In question, but also from the other DCT blocks, and by 
employing them as prediction taps, the decoded picture may be higher In picture quality than If only the quantized DCT 
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coefficients of the DCT block corresponding to the pixel block are used as the class taps. 

[0686J In view of the non-negllglble correlation between the pixel of a given pixel block and the pixels of the neigh- 
boring pixel blocks, It Is possible to Improve the picture quality of a decoded picture by employing all of the quantized 
DCT coefficients of 3x3 DCT blocks centered about the DCT block corresponding to a given pixel block as prediction 

5 taps as compared to the case ol employing only the quantized DCT coefficients of the DCT block corresponding to the 
pixel block. 

[0687] However. If all of the quantized DCT coefficients of the 3X3 DCT blocks centered about the DCT block cor- 
responding to a given pixel block are to be used as prediction taps, the number of the quantized DCT coefficients 
forming the prediction taps Is 576 (=8 x 8X3X3), with the result that the number of times of the sum-of-products process- 
10 ing to be performed by the sum-of-producte circuit 845 Is Increased. 

[0688] Thus, If quantized DCT coefficients of the 576 quantized DCT coefficients showing position relationships of 
larger correlation with respect tothe pixel of Interest are extracted and used as prediction taps, It Is possible to suppress 
the processing volume In the sum-of-products circuit 845 of Fig.80 and yetto Improve the picture quality of the decoded 
picture. 

13 [0689] In the above case, the quantized DCT coefficients showing position relationships of larger correlation with 
respect to the pixel of interest are extracted as prediction taps. However, the quantized DCT coefficients used as 
prediction taps may also be extracted from the quantized DCT coefficients of e.g., 5x5 DCT blocks centered about 
the DCT block corresponding to the given pixel block. That Is, there Is no particular limitation to which range of the 
DCT blocks are to be extracted the quantized DCT coefficients used as prediction taps. 

20 [0690] The quantized DCT coefficients of a given DCT block are obtained from the pixels of the corresponding pixel 
block, so that, In forming prediction taps for a pixel of Interest, It may be thought to be desirable to use all quantized 
DCT coefficients of the DCT block associated wtth the pixel block of the pixel of Interest. 

[0691 ] So, the pattern selection circuit 958 may be designed to generate the pattern Information in which the quan- 
tized DCT coefficients of the DCT block associated with the pixel block In question will necessarily be extracted as 
25 prediction taps, in this case, the pattern selection circuit 958 selects quantized DCT coefficients of higher correlation 
values from the eight DCT blocks neighboring to the DCT block associated with the pixel block In question, such that 
a pattern of the positions of these quantized DCT coefficients and a pattern of the positions of all quantized DCT 
coefficients of all of the DCT blocks associated with the pixel block, combined together, become the ultimate pattern 
information. 

30 [0692] Fig. 89 shows a modification of the coefficient conversion circuit 832 shown In Rg.78. in Rg.89, the parts 
corresponding to the case of Fig.80 are denoted by the same reference numerals and the corresponding explanation 
Is omitted for s Impllclty. That Is, thecoefflcient conversion circuit 832B shown In Flg.89 Is configured basically Identically 
as the corresponding device shown In Flg.80 except that there Is newly provided a dequantlzatlon circuit 871. 
[0693] In the coefficient conversion circuit 832B, shown In Flg.89, the dequantlzatlon circuit 871 is fed with block- 

6 based quantized DCT coefficients obtained on entropy decoding the encoded data from the entropy decoding circuit 
831 (Rg.78). 

[0694] Meanwhile, in the entropy decoding circuit B31 , not only the quantized DCT coefficients but also the quanti- 
zation table are obtained, as described above. In the coefficient conversion circuit 632B of Flg.89, this quantization 
table Is also fed from the entropy decoding circuit 831 to the dequantlzatlon circuit 871 . 
40 [0695] The dequantlzatlon circuit 871 dequantlzes the quantized DCT coefficients from the entropy decoding circuit 
831 In accordance with a quantization table from the entropy decoding circuit 831 to send the resulting DCT coefficients 
to the prediction tap extraction circuit 841 and to the class tap extraction circuit 842. 

[0696] Thus, the prediction tap extraction circuit 841 and the class tap extraction circuit 842 form prediction and class 
taps, not only for the DCT coefficients but also for the DCT coefficients. Subsequently, the processing similar to that 
is on Rg.80 Is performed for the DCT coefficients. 

[0697] Since the coefficient conversion circuit 832B on Rg.89 performs processing not only for the quantized DCT 
coefficiente, but also for the DCT coefficients, the tap coefficients stored in the coefficient table storage unit 844 need 
to be different from those In Flg.80. 

[0698] Thus, Rg.90 shows an illustrative structure of a tap coefficient learning device 860B responsble for the learn- 
5o ing processing of the tap coefficients, stored in the coefficient table storage unit B44 of Flg.89. Meanwhile, the parts 

or components corresponding to those shown In Flg.90 are depicted by the same reference as those used In Flg.84, 

and the corresponding explanation Is omitted for clarity. That Is, the tap coefficient learning device 860B shown In Fig. 

90 Is configured similarly to the device shown In Rg.84 except that a dequantlzatlon circuit 881 is newly provided 

downstream of the quantization circuit 863. 
55 [0699] The dequantization circuit 881 in the tap coefficient learning device 860B, shown in Rg.90, dequantizes the 

quantized DCT coefficients, output by the quantization circuit 863, in the same way as the dequantization circuit 871 

of Flg.89, and sends the resulting DCT coefficients to the prediction tap extracting circuit 864 and to the class tap 

extraction circuit 665. 
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[0700] Thus, in the prediction tap extracting circuit 864 and In the class tap extraction circuit B65, the prediction and 
tap coefficients are formed not on the quantized DCT coefficients but on the DCT coefficients. Subsequently, processing 
Is performed In the same way as In Rg.84. 

[07011 The result Is that tap coefficients are produced In which the effect of the quantization error resulting from 

5 quantization and dequantizatlon of DCT coefficients Is diminished. 

[0702] Thus, Flg.91 shows an Illustrative structure of a learning device 950B responsible for the learning processing 
of the pattern information stored in the pattern table storage unit 870 on Flg.89. Meanwhile, the parts or components 
corresponding to those shown In Flg.86 are depicted by the same reference numerals and the corresponding expla- 
nation Is omitted for clarity. That Is, the pattern learning device 950B shown In Flg.91 Is configured similarly to the 

10 device shown in Flg.86 except that a dequantizatlon circuit 8Q1 Is newly provided downstream of the quantization circuit 
953. 

[0703] In the pattern learning device 950B, the dequantizatlon circuit 891 dequantlzes the quantized DCT coefficients, 
output by the quantization circuit 953, In the same way as the dequantizatlon circuit 871 of Flg.89 or the dequantizatlon 
circuit 881 of Flg.90, and sends the resulting DCT coefficients to the summation circuit 954 andtothe class tap extraction 
«f circuit 955. 

[0704] So, the summation circuit 954 and the class tap extraction circuit 955 execute the processing not on the 
quantized DCT coefficients, but on the DCT coefficients. That Is, the summation circuit 954 executes the summation 
calculations using the DCT coefficients output by the dequantizatlon circuit 891 in place of the quantized DCT coeffi- 
cients output by the quantization circuit 953 to form class taps. Similarly, the summation circuit 954 executes the sum- 
so matlon calculations using the DCT coefficients output by the dequantizatlon circuit 891 In place of the quantized DCT 
coefficients output by the quantization circuit 953 to form class taps. Subsequently the processing similar to that In Fig. 
86 Is performed to calculate the pattern Information. 

[0705] Flg.92 shows an alternative structure of the coefficient conversion circuit 832 shown In Flg.78. Meanwhile, 
the parts or components corresponding to those shown In Rg.80 are depleted by the same reference numerals and 
25 the corresponding explanation is omitted for clarity. That is, the coefficient conversion circuit 832C shown in Rg.91 is 
configured similarly to the device shown In Rg.80 except that an Inverse DCT circuit 901 is newly provided downstream 
ofthe sum-of -products circuit 845. 

[0706] The inverse DCT circuit 901 applies inverse DCT to the output of the sum-of -products circuit 845 to decode 
the output into an output picture. Thus, In the coefficient conversion circuit 832C of Flg.92, the sum-ot-products circuit 
30 845 executes sum-of -products processing, using the q uanttzed DCT coefficients forming the prediction taps output by 
the prediction tap extraction circuit 841 and tap coefficients stored in the coefficient table storage unit 844, and outputs 
DCT coefficients. 

[0707] Thus. In the coefficient conversion circuit B32C shown In Flg.92, the quantized DCT coefficients are decoded 
Into pixel values, not by quantized DCT coefficients being subjected to sum-of-products processing with the tap coef- 
3s ficlent8 but by the quantized DCT coefficients being converted Into DCT coefficients which are then inverse DCTed by 
the Inverse DCT circuit 901 . So, the tap coefficients stored in the coefficient table storage unit 844 need to be different 
from those in Fig. 80. 

[0708] Thus. Rg.93 shows an Illustrative structure of a tap coefficient learning device 860C responsible for the learn- 
ing processing of the tap coefficients, stored in the coefficient table storage unit 844 of Flg.92. Meanwhile, the parts 

40 or components corresponding to those shown In FIg.93 are depicted by the same reference numerals as those used 
In Flg.84, and the corresponding explanation is omitted for clarity. That Is, the tap coefficient learning device 860C 
shown In Rg.93 Is configured similarly to the device shown In Rg.84 except that not the pixel values of a picture for 
learning but DCT coefficients from the DCT circuit 882. resulting from DCTing the picture for learning, are sent as 
teacher data to the normal equation summation circuit 867. 

43 [0709] Thus, the normal equation summation circuit 667 in the tap coefficient learning device 860C, shown In Fig. 
93, executes the above summation for the DCT coefficients output by the DCT circuit 862 as teacher data and for the 
quantized DCT coefficients forming the prediction tape output by the coefficient table storage unit 844 as pupil data. 
The tap coefficient decision circuit 868 solves the normal equation obtained by such summation to find tap coefficients. 
The result is that, in the learning device 860C of Rg.93. such tap coefficients for converting the quantized DCT cosf- 

50 ficients Into DCT coefficients, In which the quantization error ascribable to the quantization by the quantization circuit 
863 Is reduced or suppressed, may be found. 

[0710] In the coefficient conversion circuit 832C of Rg.92, In which the sum-of-products circuit 845 executes the sum 
of product processing, using the aforementioned tap coefficients, an output is the quantized DCT coefficients from the 
prediction tap extraction circuit 841 converted into DCT coefficients having the quantization error reduced. These DCT 
ss coefficients are inverse DCTed by the inverse DCT circuit 901 to produce a decoded picture in which the picture quality 
deterioration due to the quantization error is suppressed to a lower value. 

[071 1 ] Flg.94 shows an Illustrative structure of a pattern learning device 950C responsible for learning processing 
of the pattern Information stored In the pattern table storage unit 846 of Flg.92 and In the pattern table storage unit 870 
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of Fig.93. Meanwhile, the parts or components corresponding to those shown in Flg.86 are depicted by the same 
reference numerals and the corresponding explanation Is omitted for clartty. That Is, the pattern learning device 950C 
3hown In Fig.94 Is configured similarly to the device shown In Flg.86 except that not the pixel of a picture for learning 
output by the blocking circuit 961 but DCT coefficients output by the DCT circuit 952 are sent to the summation circuit 
5 954. 

[0712] The pattern learning device 950A of Flg.86 finds quantized DCT coefficients, showing position relationships 
of larger correlation with respect to a given pixel , by sum-of-products processing, employing quantized DCT coefflcl ents 
and tap coefficients, forming the prediction taps, In order to decode the picture, and renders the position pattern of the 
quantized DCT coefficients the pattern Information . In the pattern learning device 950C of Flg.94, the quantized DCT 
io coefficients showing a high correlation with respect to the DCT coefficients need to be found by sum-of-products 
processing employing the quantized DCT coefficients forming the prediction taps, and the tap coefficients, In order to 
produce DCT coefficients with a dhilnlshed quantization error, to find the position pattern of the quantized DCT coef- 
ficients as the pattern Information. 

[0713] Thus, In the pattern learning device 950C of Flg.94, not the pixel block obtained In the blocking circuit 951 , 

™ but the block of DCT coefficients obtained on DCTlng the plxelblock In the DCT circuit 952, are sequentially rendered 
pixel blocks of Interest and, of the DCT coefficients of one such pixel block of interest, such DCT coefficients not as 
yet rendered the DCT coefficient of interest in the raster scan nin g sequence, is sequentially made Into DCT coefficients 
of Interest. The summation circuit 954 then performs summation calculations for each class code of the DCT coefficients 
of interest output by the classification circuit 956, for finding correlation values (cross-correlation values) between the 

20 DCT coefficients of Interest and the quantized DCT coefficients output by the quantization circuit 953. 

[0714J That Is, In the learning processing by the pattern learning device 950C of Flg.94, all of the quantized DCT 
coefficients of 3X3 DCT blocks, centered about a DCT block of quantized DCT coefficients corresponding to a block 
of Interest Including a given DCT coefficient of interest, are associated with the DCT coefficients of Interest, as shown 
In Flg.95A. This operation Is carried out for ail of the blocks of DCT coefficients obtained from the picture for learning, 

25 as shown in Fig.95B, to calculate correlation values between DCT coefficients at respective positions of a given block 
of DCT coefficients and respective quantized DCT coefficients at respective positions of 3x3 DCT blocks centered 
about a DCT block corresponding to the block. Then, for each DCT coefficient of the block of the DCT coefficients, a 
position pattern of the quantized DCT coefficients exhibiting position relationships of larger correlation values with 
respect to the DCT coefficient, as Indicated by m Is rendered the pattern information. That is, In Flg.95C, a position 

so pattern of the quantized DCT coefficients showing position relationships of larger correlation with the DCT coefficient 
in a second column from left and in a first row from above of the block of DCT coefficients is represented by M, this 
position pattern being the pattern Information. 

[07151 ^ a Pixel In a number (x+1 ) column from left and In a number (y+1) row from above Is represented by A(x,y), 
and the quantized DCT coefficients of 3X3 DCT blocks, centered about the DCT block corresponding to the block 
35 Including the pixel, which quantized DCT coefficients each lie in the number (s+1 ) column from left In the number t+1 
row from above, are denoted by B(s,t), the cross-correlation values RA(x,y)B(B,t) of tne 007 coefficient A(x,y) with respect 
to the quantized DCT coefficients B(s,t) In a preset position relationship with respect to the DCT coefficient A(x,y) may 
be found as In the equations (10) to (13). 

[0716] Reverting to Flg.94, a correlative coefficient calculating circuit 957 uses the results of summation calculations 
40 performed by the summation circuit 954 to find the correlation values between the DCT coefficients and the quantized 
DCT coefficients. A pattern selection circuit 958 finds a position pattern of quantized DCT coefficients showing position 
relationships of enlarging the correlation value for use as pattern Information. 

[0717] Fig. 96 shows an alternative structure of the coefficient conversion circuit 832 shown in Fig.78. Meanwhile, 
the parts or components corresponding to those shown In Flgs.80, 89 or 92 are depicted by the same reference nu- 
45 merals and the corresponding explanation Is omitted tor clarity. That Is, the coefficient conversion circuit 832D shown 
In Flg.96 Is configured similarly to the device shown In Flg.80 except that an Inverse DCT circuit 871 Is newly provided 
as In Flg;89 and that the Inverse DCT circuit 901 Is also provided as In Flg.92. 

[0718] Thus, In the prediction tap extraction circuit 841 and In the class tap extraction circuit 842, In the coefficient 
conversion circuit 832D shown in Fig. 96, the prediction and class taps are formed not on the quantized DCT coefficients 

so but on the DCT coefficients. Moreover, In the coefficient conversion circuit 832D shown in Flg.96, the sum-of-products 
circuit 845 performs sum-of-products processing, using the DCT coefficients forming the prediction taps output by the 
prediction tap extraction circuit 841 and tap coefficients stored In the coefficient table storage unit 844, to produce DCT 
coefficients having a reduced quantization error. These DCT coefficients are output to the inverse DCT circuit 901 . 
[0719] Thus, FIg.97 shows an illustrative structure of a tap coefficient learning device 860D responsible for learning 

55 processing of the tap coefficients, stored in the coefficient table storage unit 844 of Fig.96. Meanwhile, the parts or 
components corresponding to those shown in Fig.97 are depicted by the same reference as those used in Figs.84. 90 
and 93, and the corresponding explanation Is omitted for clarity. That Is, the tap coefficient learning device 860D shown 
In Ffg.97 Is configured similarly to the device shown In Flg.84 except that a dequantlzatlon circuit 881 Is newly provided 
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as In Flg.90 and that not the pixel values of a picture for learning but DCT coefficients from the DCT circuit 862, resulting 
from the DCTlng of the picture for learning, are sent as teacher data to the normal equation summation circuit 867. 
[0720] Thus, the normal equation summation circuit 8S7 In the tap coefficient learning device 860D, shown In Fig. 
97, executes the above summation for the DCT coefficients output by the DCT circuit 662 as teacher data and for the 

s quantized DCT coefficients (quantized and dequantlz ed data) forming the prediction taps output by the coefficient table 
storage unit 844 as pupil data. The tap coefficient decision circuit 868 solves the normal equation obtained by such 
summation to find tap coefficients. The result Is that, In the learning device 860D of Fig.97» such tap coefficients for 
converting the quantized and dequantlzed DCT coefficients Into DCT coefficients, In which the quantization error as- 
crlbable to the quantization and dequantteatlon, Is reduced or suppressed, may be found. 

10 [0721] Flg.98 shows an illustrative structure of a pattern learning device 950D responsible for learning processing 
of the pattern Information stored In the pattern table storage unit 846 of Flg.96 and In the pattern table storage unit 870 
of Flg.97. Meanwhile, the parts or components corresponding to those shown In Flgs.86, 91 and 94 are depicted by 
the same reference numerals and the corresponding explanation is omitted for clarity. That Is, the pattern learning 
device 950D shown In Flg.98 Is configured similarly to the device shown In Flg.86, except that, as In Flg.94, not the 

" pixel of a picture for learning output by the blocking circuit 951 but DCT coefficients output by the DCT circuit 952 are 
sent to the summation circuit 954. 

[0722] Thus, In the summation circuit 954 of the pattern learning device 950D of Fig. 98, not the pixel blocks obtained 
in the blocking circuit 951 but the blocks of DCT coefficients obtained on DCTlng the pixel blocks In the DCT circuit 
952, are sequentially made into blocks of interest, and the DCT coefficients of the pixels of interest not as yet made 

20 into the DCT coefficients of Interest In the raster scanning sequence are made Into DCT coefficients of Interest. The 
summation circuit 954 then executed summation processing, from one class code of the DCT coefficients of Interest 
output by the classification circuit 956 to another, in order to find a value of correlation (cross-correlation value) between 
the DCT coefficients of Interest and the quantized and dequantlzed DCT coefficients output from the Inverse quanti- 
zation circuit 891 . Using the result of the summation calculations, executed by the summation circuit 954, the correlation 

25 coefficient calculating circuit 957 finds correlation values between the DCT coefficients and the quantized and dequan- 
tlzed DCT coefficients. The pattern selection circuit 958 finds a position pattern of quantized and dequantlzed DCT 
coefficients showing the position relationships of enlarging the correlation value. 

[0723] Fig. 99 shows another illustrative structure of the coefficient conversion circuit 832 of Fig. 78. Meanwhile, parts 
or components of Fig. 99 corresponding to those shown in Flg.80 are indicated by the same reference numerals and 

30 the corresponding explanation Is sometimes omitted for clarity. That Is, the coefficient conversion circuit 832E shown 
in Fig.99 Is basically configured simflarfy to the circuit shown in Fig.80 except that the tap coefficient conversion circuit 
832E shown In Fig.99 is destitute of the class tap extraction circuit 842 or the classification circuit 843. 
[0724] So, the coefficient conversion circuit 832E shown In Flg.99 lacks In the concept of the class, which may be 
said to be equivalent to the number of the classes being one. Thus, only one class of the tap coefficients Is stored In 

3ff the coefficient table storage unit 844 such that processing Is executed using this one class of the tap coefflclentB. 

[0725] That Is, In the coefficient conversion circuit 832E on F!g.99 t the tap coefficients memorized In the coefficient 
table storage unit 844 differ from those shown in Fig.80. 

[0726] Flg.1 00 shows an illustrative structure or the learning device 860 E for executing the processing of learning 
the tap coefficients to be stored In the coefficient table storage unit 844 of Flg.99. Meanwhile, the parts or components 
4° of Flg.100correspondlngtothoseshown In Flg.84are Indicated by the same reference numerals andthecorrespondlng 
explanation is sometimes omitted for clarity. That is t the tap coefficient conversion circuit 860E shown in Fig. 100 is 
basically configured similarly to the circuit shown In Flg.64 except that the tap coefficient conversion circuit B60E shown 
in Fig.1 00 is destitute of the class tap extraction circuit 865 or the classification circuit 866. 

[0727] Thus, in the tap coefficient learning device 860E of Fig. 100. the above-described summation is carried out 
« without dependency on the class In the normal equation summation circuit 267 from one pixel position mode to another. 
The tap coefficient decision circuit 868 solves the normal equations generated from one pixel position mode to another 
to And the tap coefficients. 

[0728] So, the coefficient conversion circuit 832E shown in Fig.99 has only one class so that only one class of the 
pattern information is stored in the pattern table storage unit 846 on Rg.99 or in the pattern table storage unit 870 of 
so Flg.1 00. 

[0729] Flg.1 01 shows an Illustrative structure of a pattern learning device 960 E for executing the processing of learn- 
ing the pattern Information to be stored In the pattern table storage unit 846 of Flg.99 and In the pattern table storage 
unit 870 of Rg.1 00. Meanwhile, the parts or components of Flg.1 01 corresponding to those shown In Fig.86 are indi- 
cated by the same reference numerals and the corresponding explanation is sometimes omitted for clarity. That Is, the 
55 pattern learning device 960E shown in Fig. 101 is basically configured similarly to the circuit shown in Fig. 86 except 
that ft is destitute of the class tap extraction circuit 955 or the classification circuit 956. 

[0730} Thus, In the pattern learning device 960 E of Fig. 101 , the above-described summation Is carried out without 
dependency on the class in the summation circuit 954 from one pixel position mode to another. The correlative coef- 
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fldent calculating circuit 957 also finds the pattern Information from one pixel position mode to another to rind the 
pattern Information based on the correlation value obtained In the correlative coefficient calculating circuit 957. 
[0731 ] In the coefficient conversion circuit 832A of Flg.80, the class-based pattern Information Is stored In the pattern 
table storage unit 846, and the prediction tap is formed using the pattern information of the class corresponding to the 
s class code output by the classification circuit 843. Alternatively, the one-class pattern Information obtained In the learn- 
ing device 960E of Fig. 101 may be stored In the pattern table storage unit 846 to form a prediction tap. using the pattern 
Information, without dependency on the class. 

[0732] The above-described sequence of operations may be performed on a hardware or a software. In the latter 
case, the program forming the software Is Installed In e.g., a general-purpose computer. 
10 [0733] Fig. 1 02 shows an Illustrative structure of a computer on which Is installed the program adapted for executing 
the above-described sequence of operations. 

[0734] The program may be pre-recorded on a head disc 1205 or a ROM 1203, as a recording medium enclosed In 
a computer 1 000. 

[0735] Alternatively, the program may be transiently or permanently stored (recorded) In a removable recording me- 
is dlum 1211 , such as a floppy disc, CD-ROM (Compact Disc Read Only Memory), MO (Magneto-Optical) disc, a DVD 
(Digital Versatile Disc), a magnetic disc or on a semiconductor memory. This removable recording medium 1211 may 
be furnished as so-called package software. 

[0736] Meanwhile, the program may not only be Installed from the removable recording medium 1 21 1 to the computer, 
but also may be transferred from a downloading site over an artificial satellite for digital satellite broadcast over a radio 
20 path to the computer, or transferred through a network, such as LAN (Local Area Network) or the Internet, over a cable 
to the computer, which then receives the program, thus transferred, In a communication unit 1208 to Install the program 
In an enclosed hard disc 1205. 

[0737] The computer 1 000 has a built-in CPU (Central Processing Unit) 1 202 to which Is connected an Input output 
Interface over a bus 1201. If a command Is Input to the CPU 1202 through the Input output Interface 1210 by the user 

25 acting on an input unit 1207, such as keyboard, mouse or a microphone, the program stored in the ROM (Read Only 
Memory) 1203 is run. Alternatively, the CPU 1202 loads a program stored In the hard disc 1205, a program transferred 
from the satellite or network, received by the communication unit 1208 and Installed on a hard disc 1 205, or a program 
read out from the removable disc 1211 loaded on a drive 1 209 and Installed on the hard disc 1 205, on the RAM (Random 
Access Memory) 1204, to execute the so loaded program. The processed results are output by the CPU 1202 through 

30 the Input output Interface 121 0 via an output unit 1206 designed as an LCD (liquid crystal display) or a loudspeaker, 
transmitted from a communication unit 1208 or recorded on the hard disc 1205. 

[0738] In the present specification , the processing steps stating the program designed for the computer to carry out 
a variety of processing operations are not necessarily processed chronologically In a sequence stated In the flowchart, 
but the processing also may Include processing carried out In parallel or batch-wise, such as parallel processing or 
3ff processing by an object. 

[0739] The program may be processed by one computer or by a plural number of computers In a distributed fashion. 
The program may also be transferred to and executed by a remote computer. 

[0740] The above-described respective embodiments are directed to picture data. The present Invention may, how- 
ever, be applied to e.g., speech data. 

[0741 ] The above-described respective embodiments are directed to a picture JPEG encoded for compression en- 
coding of a still picture. The present invention may, however, be applied to a picture encoded In accordance with M PEG 
for compression encoding a moving picture. 

[0742] The above-described embodiments are directed to the decoding of the J PEG encoded data for DCT process- 
ing. The present invention may. however, be applied to decoding or conversion of data transformed from block to block, 
that Is from one preset unit to another, by orthogonal transform or frequency transform. That Is, the present Invention 
may be applied to decoding sub-band encoded or Fourier-transformed data orto transforming the data Into data lowered 
In e.g., quantization error. 

[0743] In the above-described embodiments, tap coefficients used for decoding are prestored In the decoder 22. 
These tap coefficients may, however, be included in the encoded data and furnished to the decoder in this state. The 
so same may be said of<the pattern information. 

[0744] Although decoding and transform are carried out by linear predictive calculations employing tap coefficients, 
the decoding or transform may also be carried out by higher order predictive calculations, such as order-two predictive 
calculations. 

[0745] Although the predictive tap is comprised of quantized DCT coefficients of a DCT block corresponding to the 
ss pixel bbck of interest and a plural number of neighboring DCT blocks, cluster taps may also be formed in s milar manner. 
[0746] According to the present invention, as described above, converted data used in predictive calculations for 
decoding data ol the data block of Interest are extracted at least from the converted block corresponding to the data 
block other than the data block of Interest, for use as prediction taps. Using the tap coefficients and the prediction taps, 
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preset predictive calculations are carried out to decode the converted data Into original data efficiently. 
[0747] According to the present Invention, teacher data as teacher are at least orthogonal or frequency transformed 
to form pupil data as pupils. Pupil data used for predictive calculations for decoding the teacher data of a teacher block 
of Interest in a teacher block are extracted from a pupil block corresponding to the teacher block other than the teacher 
block of Interest for use as prediction tap. Learning Is carried out such as to statistically minimize the prediction errors 
of the predicted values of teacher data obtained on predictive calculations employing the tap and prediction coefficients. 
Using the tap coefficients, thus found, orthogonal or frequency transformed data can be decoded efficiently. 
[0743] Moreover, according to the present Invention, using first and second data for learning the extraction pattern, 
the correlation between the first data In the preset position relationships with respect to the second data at a preset 
position Is found from one position relationship to another to set an extraction pattern of the first data to be used as 
prediction taps used for predictive calculations for the second data. Predictive calculations may be carried out using 
the first data extracted In accordance with the extraction pattern to convert e.g., the first data efficiently Into the second 
data. 



Claims 

1 . A data processing apparatus for processing converted data obtained on orthogonal or frequency transform, com- 
prising: 

acquisition means for acquiring tap coefficients as found by learning; and 

decoding means for decoding said converted data to original data by preset predictive calculations employing 
said tap coefficients and said converted data. 

2. The data processing apparatus according to claim 1 wherein said decoding means perform linear predictive cal- 
culations using said tap coefficients and the converted data to decode the converted data to original data. 

3. The data processing apparatus according to claim 1 further comprising: 

storage means for storing said tap coefficients; 

said acquisition means acquiring said tap coefficients from said storage means. 

4. The data processing apparatus according to claim 1 wherein said converted data Is obtained on orthogonal or 
frequency transforming said original data followed by quantization. 

5. The data processing apparatus according to claim 4 further comprising: 

dequantlzlng means for dequantlzlng said converted data; 

said decoding means decoding the dequantlzed converted data into said original data. 

6. The data processing apparatus according to claim 1 wherein said converted data Is said original data at least 
discrete cosine transformed. 

7. The data processing apparatus according to claim 1 further comprising: 

predictive tap extraction means for extracting said converted data used along with said tap coefficients for 
predicting data of Interest in said original data and for o inputting the data as prediction taps; 
said decodingmeans executing predictive calculations using said prediction and tap coefficients. 

8. The data processing apparatus according to claim 7 further comprising: class tap extraction means for extracting 
said converted data used for assigning said data of Interest to one of a plural number of classes and for outputting 
the data as class tap; and 

classifying means for finding the class of said data of Interest based on said class tap; 
said decoding means performing predictive calculations using said predictive tap and said tap coefficients 
corresponding to the class of said data of interest. 

9. The data processing apparatus according to claim 1 wherein said converted data Is said original data at least 
orthogonal or frequency transformed from one preset unit to another; 
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said decoding means decoding said converted data to said original data from one said preset unit to another. 

10. The data processing apparatus according to dalm 1 wherein said tap coefficients are obtained on learning such 
as to statistically minimize a prediction error of a prediction value of said original data , obtained on preset predictive 

5 calculations, using said tap coefficients and said converted data. 

1 1 . The data processing apparatus according to claim 1 wherein said original data Is data of a moving picture or a still 
data 

io 12. A data processing method for processing converted data obtained on orthogonal or frequency transform, compris- 
ing the steps of: 

acquiring tap coefficients as found by learning; and 

decoding saJd converted data to original data by preset predictive calculations employing said tap coefficients 
is and said converted data. 

13. A recording medium having recorded thereon a program for having a computer execute data processing of process- 
ing converted data obtained at least on orthogonal or frequency transform; said program Including the steps of: 

20 acquiring tap coefficients as found by learning; and 

decoding said converted data to original data by preset predictive calculations employing said tap coefficients 
and said converted data. 

14. A data processing apparatus for learning tap coefficients used for decoding converted data, obtained at least on 
25 orthogonal transform or frequency transform, by predictive calculations, comprising: 

generating means for generating pupil data, as pupil, by at least orthogonal or frequency transforming teacher 
data, as teacher; and 

learning means for effecting learning so that prediction errors of prediction values of said teacher data obtained 
so on predictive calculations using said tap coefficients and the pupil data will be statistically minimum, to find 

said tap coefficients. 

15. Tne data processing apparatus according to claim 14 wherein said learning means executes learning so that a 
prediction error of the prediction value of said teacher data obtained on linear predictive calculations using said 

35 tap coefficients and the pupil data will be statistically minimum. 

16. The data processing apparatus according to clarn 14 wherein said generating means generate said pupil data by 
orthogonal or frequency transform followed by quantization of said teacher data. 

^0 17. The data processing apparatus according to dafrn 14 wherein said generating means generate said pupil data by 
orthogonal or frequency transform, followed by quantization and dequantlzatlon, in this order, of said teacher data. 

18. The data processing apparatus according to claim 14 wherein said generating means generate said pupil data al 
least by discrete cosine transforming said teacher data. 

45 

1 9. The data processing apparatus according to claim 1 4 further comprising: 

prediction tap extracting means for extracting said pupil data used along with said tap coefficients for predicting 
the teacher data of interest in said teacher data and for outputting the extracted pupil data as prediction taps; 
so said learning means executing learning so that a prediction error of the prediction value of said teacher data 

obtained on linear predictive calculations using said prediction taps and tap coefficients will be statistically 
minimum. 

20. The data processing apparatus according to claim 19 further comprising: 

55 

class tap extracting means for extracting said pupil data used for assigning said teacher data of interest to 

one of a plural number of classes and for outputting the data as class tap; and 

classifying means forfindlng the class of said teacher data of Interest based on said class tap; 
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said learning means effecting learning so that a prediction error of the prediction value or said teacher data 
obtained on predictive calculations using said prediction taps and said tap coefficients will be statistical^ min- 
imum to find said tap coefficients on the class basis. 

5 21 . The data processing apparatus according to claim 14 wherein said generating means performs at least the or- 
thogonal transform or frequency transform from one said preset unit to another to generate said pupil data. 

22. The data processing apparatus according to claim 1 4 wherein said teacher data is a moving picture or a still picture. 

to 23. A data processing method for learning tap coefficients used for decoding converted data, obtained at least on 
orthogonal transform or frequency transform , by predictive calculations, comprising the steps of: 

generating pupil data , as pupil, by at least orthogonal or frequency transforming teacher data, as teacher; and 
effecting learning so that prediction errors of the prediction values of said teacher data obtained on predictive 
w calculations using said tap coefficients and the pupil data will be statistically minimum to find said tap coeffi- 

cients. 

24. A recording medium having recorded thereon a program for having a computer execute data processing of process- 
ing converted data obtained at (east on orthogonal or frequency transform; said program including the steps of: 

20 

generating pupil data, as pupil, by at least orthogonal or frequency transforming teacher data, as teacher; and 
effecting learning so that prediction errors of the prediction values of said teacher data obtained on predictive 
calculations using said tap coefficients and the pupil data will be statistically minimum to find said tap coeffi- 
cients. 

25 

25. A data processing apparatus for processing converted data obtained on orthogonal or frequency transform, com- 
prising: 

acquisition means for acquiring tap coefficients as found by learning; and 
30 calculating means for executing preset predictive calculations using said tap coefficients and the converted 

data to decode said converted data to original data and to produce processed data which is said original data 
subjected to preset processing. 

26. The data processing apparatus according to claim 25 wherein said calculating means execute linear predictive 
as calculations using said tap coefficients and the converted data. 

27. The data processing apparatus according to claim 25 further comprising: 

storage means for storing said tap coefficients; 
40 said acquisition means acquiring said tap coefficients from said storage means. 

26. The data processing apparatus according to claim 25 wherein said converted data Is obtained on orthogonal trans- 
forming or frequency transforming and further quantizing said original data. 

45 29. The data processing apparatus according to claim 28 further comprising: 

dequantizatlon means for dequantlzlngsald converted data; 

said calculating means performing predictive calculations using the deq uantized converted data. 

50 30. The data processing apparatus according to claim 25 wherein said converted data is obtained on at least discrete 
cosine transforming said original data. 

31 . The data processing apparatus according to claim 25 further comprising: 

55 prediction tap extraction means for extracting said converted data, among said processed data, used along 

with said tap coefficients for predicting data of interest and for outputting the extracted data as prediction tap; 
said calculating means performing predictive calculations using said prediction tap and the tap coefficients. 
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32. The data processing apparatus according to claim 31 further comprising: 

class tap extraction means for extracting said converted data used for assigning said data ol Interest to one 
of a plural number of classes and for outputtlng the data as class tape; and 
5 classification means for finding the class of said data of Interest, based on said class taps, by way of classifi- 

cation; 

said calculating means performing predictive calculations using said prediction taps and the tap coefficients 
corresponding to the class of the data of Interest. 

10 33. The data processing apparatus according to claim 25 wherein said calculating means perform said preset predictive 
calculations to produce said processed data which Is said original data processed for Improving the data quality. 

34. The data processing apparatus according to claim 25 wherein said tap coefficients are obtained on effecting learn- 
ing, using said tap coefficients and the converted data, so that a prediction error of the predicted value of the 

is processed data obtained on executing preset predictive calculations will be statistically minimum. 

35. The data processing apparatus according to claim 25 wherein said original data is picture data of a moving picture 
or a still picture. 

20 36. The data processing apparatus according to claim 35 wherein said calculating means perform said preset predictive 
calculations to produce said processed data which Is said original data processed for Improving the picture quality. 

37. The data processing apparatus according to claim 35 wherein said calculating means produces said processed 
data which Is said picture data Improved In temporal or spatial resolution. 

25 

38. A data processing method lor processing converted data obtained at least on orthogonal or frequency transform, 
comprising the steps ol: 

acquiring tap coefficients as found by learning; and 
30 executing preset predictive calculations using said tap coefficients and the converted data to decode said 

converted data to original data and to produce processed data which is said original data subjected to preset 
processing. 

39. A recording medium having recorded thereon a program for having a computer execute data processing of process- 
3s ing converted data obtained at least on orthogonal or frequency transform, said program Including the steps of: 

acquiring tap coefficients as found by learning; and 

executing preset predictive calculations using said tap coefficients and the converted data to decode said 
converted data to original data and to produce processed data which Is said original data subjected to preset 
40 processing. 

40. A data processing apparatus for decoding converted data obtained at least on orthogonal or frequency transform 
and for learning tap coefficients used for predictive calculations for processing decoded results in a preset fashion , 
comprising: 

45 

quasl-teacher data generating means for processing teacher data as teacher, based on said preset processing, 
to produce quasi -teacher data; 

pupil data generating means for generating pupil data as pupils on at least orthogonal or frequency transform- 
ing said quasi-teacher data; and 
so learning means tor effecting learning for statistically minimizing a prediction error of the predicted value of said 
teacher data obtained on predictive calculations employing said tap coefficients and the pupil data to find said 
tap coefficients. 

41. The data processing apparatus according to claim 40 wherein said learning means effects learning so that the 
55 prediction error ofthe prediction value of said teacher data obtained on performing linear predictive calculations 

employing said tap coefficients and the pupil data will be statistically minimum. 

42. The data processing apparatus according to claim 40 wherein said pupil data generating means generates said 
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pupil data by orthogonal or frequency transforming and further quantizing said quasl-teacher data. 

43. The data processing apparatus according to claim 40 wherein said learning means generates said pupil data by 
orthogonal or frequency transforming, quantizing and further dequantizlng said quasl-teacher data. 

44. The data processing apparatus according to claim 40 wherein said pupil data generating means generates said 
quasl-teacher data on at least discrete cosine transforming said quasl-teacher data. 

45. The data processing apparatus according to claim 40 further comprising: 

prediction tap extracting means for extracting said pupil data used along with the tap coefficients for predicting 
the teacher data of interest In said teacher data and for outputUng the extracted data as class taps; 
said learning means effecting learning so that a prediction error of the prediction value of said teacher data 
obtained on performing linear predictive calculations employing said prediction taps and the tap coefficients 
will be statistically minimum. 

48. The data processing apparatus according to claim 45 comprising: 

class tap extraction means for extracting said pupil data used for assigning said teacher data of interest to 
one of a plurality of classes, by way of classification, and for outputtlng the extracted data as class taps; and 
classification means for finding the class of said teacher data of Interest based on said class taps by way of 
classification; 

said learning means effecting learning for statistically minimizing a prediction error of the prediction value of 
said teacher data obtained on performing predictive calculations using said prediction taps and the tap coef- 
ficients corresponding to the class of said teacher data of interest to find the class-based tap coefficients. 

47. The data processing apparatus according to claim 40 wherein said pupil data generating means at least orthogonal 
or frequency transforms said quasi-teacher data every preset unit to generate said pupil data. 

48. The data processing apparatus according to claim 40 wherein said quasl-teacher data generating means generates 
said quasi-teacher data by processing said teacher data in a manner of deteriorating the data quality. 

49. The data processing apparatus according to claim 40 wherein said teacher data Is a moving picture or a stlil picture. 

50. The data processing apparatus according to claim 49 wherein said quasi-teacher data generating means generates 
said quasl-teacher data by processing said picture data in a manner of deteriorating the picture quality. 

51 . The data processing apparatus according to claim 49 wherein said quasl-teacher data generating means generates 
said quasl-teacher data which Is said picture data deteriorated In temporal or spatial resolution of said picture data. 

52. A data processing method for decoding converted data obtained at least on effecting orthogonal or frequency 
transform and for learning tap coefficients used for predictive calculations for processing decoded results In a 
preset fashion; comprising the steps of: 

processing teacher data as teacher, based on said preset processing, to produce quasl-teacher data; 
generating pupil data as pupils on at least orthogonal or frequency transforming said quasl-teacher data; and 
effecting learning for statistically minimizing the prediction error of the predicted value of said teacher data 
obtained on predictive calculations employing said tap coefficients and the pupil data to find said tap coeffi- 
cients. 

53. A recording medium having recorded thereon a program for having a computer execute data processing of de- 
coding converted data obtained at least on orthogonal or frequency transform and of learning tap coefficients used 
for predictive calculations for processing decoded results in a preset fashion; said program Including the steps of: 

processing teacher data as teacher, based on said preset processing, to produce quasi-teacher data; 
generating pupil data as pupils on at least orthogonal or frequency transforming said quasi-teacher data; and 
effecting learning for statistically minimizing the prediction error of the predicted value of said teacher data 
obtained on predictive calculations employing saJd tap coefficients and the pupil data to find said tap coeffl- 
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dents. 

54. A data processing apparatus for processing encoded data Including converted data obtained on at least orthogonal 
or frequency transform and preset subsidiary Information, and for decoding said converted data Into original data, 
comprising: 

classification means for assigning data of Interest In said original data to one of a plurality of classes, based 
on said subsidiary Information, by way of classification; 

acquisition means for acquiring tap coefficients associated with the classes of said data of Interest, In class- 
based tap coefficients as found on learning; and 

decoding means for executing preset predictive calculations, using the converted data and said tap coefficients 
associated with the classes of said data of Interest, to decode said converted data Into original data. 

55. The data processing apparatus according to claim 54 wherein said decoding means execute linear predictive 
calculations, using said converted data and the tap coefficients, to decode said converted data to original data. 

56. The data processing apparatus according to claim 54 further comprising: 

storage means for storing said class-based tap coefficients; 

said acquisition means acquiring the tap coefficients associated with the class of said data of Interest from 
said storage means. 

57. The data processing apparatus according to claim 54 wherein said converted data is acquired on orthogonal or 
frequency transforming and further quantizing said original data. 

58. The data processing apparatus according to claim 57 further comprising: 

dequantizing means for dequantizlng said converted data; 

said decoding means decoding said dequantized converted data into said original data. 

59. The data processing apparatus according to claim 57 wherein said subsidiary information is a quantization table 
used In quantizing said original data. 

60. The data processing apparatus according to claim 54 wherein said converted data Is said original data at least 
discrete cosine transformed. 

61 . The data processing apparatus according to claim 54 further comprising: 

prediction tap extracting means for extracting said converted data used along with said tap coefficients In 

predicting said data of Interest and for outputtlng the extracted data as prediction tap; 

said decoding means executingpredlctive calculations using said prediction taps and the tap coeff Iclents. 

62. The data processing apparatus according to claim 54 further comprising: 

class tap extraction means for extracting said converted data used for assigning said data of Interest to one 
of a plurality of classes to output the extracted data as class tap; 

said classification means finding the class of said data of interest based on said subsidiary Information and 
the class taps. 

63. The data processing apparatus according to claim 54 wherein said converted data Is said original data at least 
orthogonal transformed or frequency transformed every preset unit; and wherein 

said decoding means decodes said converted data Into said original data every said preset unit. 

64. The data processing apparatus according to claim 54 wherein said tap coefficients are obtained on effecting learn- 
ing, using said tap coefficients and the converted data, so that a prediction error of a predicted value of said original 
data obtained on preset predictive calculations will be statistically minimum. 

65. The data processing apparatus according to dalm 54 wherein said original data Is data of a moving picture or a 
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still picture. 

66. A data processing method for processing encoded data Including converted data, obtained on at least orthogonal 
or frequency transform, and preset subsidiary Information, and for decoding said converted data into original data, 

5 comprising the steps of: 

assigning data of interest in said original data to one of a plurality of classes, based on said subsidiary infor- 
mation, by way of classltlcatlon; 

acquiring tap coefficients associated with the classes of said data of Interest, In class-based tap coefficients 
10 as found on learning; and 

executing preset predictive calculations, using the converted data and said tap coefficients associated with 
the classes of said data of Interest, to decode saJd converted data Into original data. 

67. A recording medium having recorded thereon aprogram for having a computer execute data processing of process- 
fng encoded data Including converted data, obtained on at least orthogonal or frequency transform, and preset 
subsidiary Information, and for decoding said converted data into original data, said program including the steps of: 

assigning data of interest In said original data to one of a plurality of classes, based on said subsidiary Infor- 
mation, by way of classification; 
20 acquiring tap coefficients associated with the classes of said data of Interest, In class-based tap coefficients 

as found on learning; and 

executing preset predictive calculations, using the converted data and said tap coefficients associated with 
the classes of said data of Interest, to decode said converted data Into original data 

25 68. A data processing apparatus for learning tap coefficients used for decoding converted data, obtained at least on 
orthogonal transform or frequency transform, by predictive calculations, comprising: 

generating means for generating pupil data, as pupil, at least on orthogonal or frequency transforming teacher 
data as teacher, 

30 classification means for assigning teacher data of Interest In said teacher data to one of a plurality of classes, 

based on the preset subsidiary information used in generating said pupil data in said generating means; and 
learning means for effecting learning so that prediction errors of the prediction values of said teacher data 
obtained on predictive calculations using the tap coefficients corresponding to the classes of said teacher data 
of Internet and pupil data will be statistically minimum to find class-based tap coefficients. 

36 

69. The data processing apparatus according to claim 68 wherein said learning means effects learning so that a pre- 
diction error of the prediction value of said teacher data obtained on iinear predictive calculations using said tap 
coefficients and the pupil data will be statistically minimum. 

40 70. The data processing apparatus according to dalm 68 wherein said generating means generates said pupD data 
on orthogonal or frequency transforming said teacher data and further on quantizing the converted teacher data. 

71. The data processing apparatus according to claim 70 wherein said subsidiary information is a quantization table 
used in quantizing said teacher data. 

43 

72. The data processing apparatus according to claim 68 wherein said generating means generates said pupil data 
on orthogonal or frequency transforming said teacher data and further on quantizing and dequantlzing the data. 

73. The data processing apparatus according to claim 68 wherein said generating means generates said pupil data 
50 at least on discrete cosine transforming said teacher data. 

74. The data processing apparatus according to claim 68 further comprising: 

prediction tap extracting means for extracting said pupil data used along with said tap coefficients for predicting 
55 said teacher data of interest and for outputting the extracted data as prediction tap; 

said learning means effecting learning so that a prediction error of the prediction value of said teacher data 
obtained on linear predictive calculations using said tap coefficients and the pupil data will be statistically 
minimum. 
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75. The data processing apparatus according to claim 68 further comprising: 

class tap extracting means for extracting said pupil data used for assigning said teacher data of Interest to 
one of a plurality of classes by way of classification and for outputtlng the extracted data as class taps; 
5 said classification means finding the class of said teacher data of Interest based on said s ubsldlary Information 

and said class taps. 

78. The data processing apparatus according to claim 68 wherein said generating means generates said pupil data 
at least on orthogonal or frequency transforming said teacher data every preset unit. 

77. The data processing apparatus according to claim 68 wherein said teacher data Is a moving picture or a still picture. 

78. A data processing method for learning tap coefficients used for decoding converted data, obtained at least on 
orthogonal transform or frequency transform, by predictive calculations, comprising the steps of: 

13 

generating pupil data, as pupil, at least on orthogonal or frequency transforming teacher data as teacher; 
asstgnlng teacher data of interest In said teacher data to one of a plurality of classes based on the preset 
subsidiary Information used In generating said pupil data In said generating means; and 
effecting learning so that a prediction error of the prediction value of said teacher data obtained on predictive 
20 calculations using the tap coefficients corresponding to the classes of said teacher data of Interest and pupil 

data will be statistically minimum, to find class-based tap coefficients. 

79. A recording medium having recorded thereon a program for having a computer execute data processing of learning 
tap coefficients used for decoding converted data, obtained at least on orthogonal transform or frequency transform, 

25 by predictive calculations, said program including the steps of: 

generating pupil data, as pupil, at least on orthogonal or frequency transforming teacher data as teacher; 
assigning teacher data of Interest In said teacher data to one of a plurality of classes based on the preset 
subsidiary Information used in generating said pupil data In said generating means; and 
30 effecting learning so that a prediction error of the prediction value of said teacher data obtained on predictive 

calculations using the tap coefficients corresponding to the classes of said teacher data of interest and pupil 
data will be statistically minimum, to find class-based tap coefficients. 

A data processing apparatus for finding, from converted data based on a preset block as a unit, obtained at least 
on orthogonal or frequency transforming data in terms of said preset block as a unit, a predicted value of processed 
data which Is said converted data processed In a desired fashion; 

acquisition means for acquiring tap coefficients as found on learning; 

class tap extraction means for extracting such data of said converted data, used for assigning processed 
data of Interest to one of a plurality of classes, from at least a block other than the block associated with said 
processed data of Interest, by way of classlf Ication, and for outputtlng the extracted data as class taps; 

classification means for finding a class of said processed data of Interest based on said class tap; and 
calculating means for executing preset predictive calculations, using said tap coefficients of the class of said 
processed data of interest, and said data of said converted data, to find prediction values of said processed data 
of interest. 

The data processing apparatus according to claim 80 wherein said calculating means uses said tap coefficients 
and the converted data to effect linear predictive calculations. 

The data processing apparatus according to claim 80 further comprising: 

storage means for storing said tap coefficients; 

said acquisition means acquiring said tap coefficients from said storage means. 

83. The data processing apparatus according to claim 80 wherein said converted data Is said data at least discrete 
cosine transformed. 

84. The data processing apparatus according to claim 83 wherein said classification means finds the class of said 
processed data of Interest based on the power of DC or AC components of discrete cosine transformed data which 
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Is said converted data as said class tap. 

85. The data processing apparatus according to claim 60 wherein said class tap extraction means extracts said con- 
verted data as said class tap from blocks neighboring to a block associated with processed data of interest. 

5 

86. The data processing apparatus according to dalm 80 wherein said class tap extraction means extracts said con* 
verted data as said class taps even from the block corresponding to the processed data of Interest. 

87. The data processing apparatus according to claim 60 wherein said tap coefficients are obtained on effecting learn - 
10 |ng so that a prediction error of a prediction value obtained on performing preset predictive calculations using said 

tap coefficients and the converted data will be statistically minimum. 

86. Trie data processing apparatus according to dalm 80 wherein said data is picture data of a moving picture or a 
still picture. 

13 

89. A data processing method for finding, from converted data based on a preset block as a unit, obtained at least on 
orthogonal or frequency transforming data in terms of said preset block as a unit, a predicted value of processed 
data which Is said converted data processed In a desired fashion, comprising the steps of: 

20 acquiring tap coefficients as found on learning; 

extracting such data of said converted data, used for assigning processed data of Interest to one of a plurality 

of classes, from at least a block other than the block associated with said processed data of interest, by way 

of classification, and for outputtlng the extracted data as class taps; 

finding a class of said processed data of Interest based on said class tap; and 
25 executing preset predictive calculations, using said tap coefficients of the class of said processed data of 

Interest, to find a prediction value of said processed data of Interest. 

90. A recording medium having recorded thereon a program for having a computer execute data processing of finding, 
from such data of converted data based on a preset block as a unit, obtained at least on orthogonal or frequency 

30 transforming data In terms of said preset block as a unit, a predicted value of processed data which is said converted 

data processed in a desired manner, said program Including 
acquiring tap coefficients as found on learning; 

extracting such data of said converted data, used for assigning processed data of interest to one of a plurality 
of classes, from at least a block other than the block associated with said processed data of Interest, by way of 
3& classification, and for outputtlng the extracted data as class taps; 

finding a class of said processed data of Interest based on said class tap; and 

executing preset predictive calculations, using said tap coefficients of the class of said processed data of 
interest, to find a prediction value of said processed data or Interest. 

4° 91 . A data processing apparatus for learning, from converted data based on a preset block as a unit, obtained at least 
on orthogonal or frequency transforming data in terms of said preset block as a unit, tap coefficients used for 
finding, by predictive calculations, processed data which Is said converted data processed In a desired fashion, 
comprising: 

+s generating means for generating said block-based converted data as pupil data as pupil In learning; 

class tap extraction means for extracting pupil data used for assigning teacher data of Interest to one of a 
plurality of classes, at least from a block other than a block corresponding to said teacher data of interest, with 
said processed data as teacher data as teacher in learning, and for outputtlng the extracted data as class taps; 
classification means for finding a class of said teacher data of interest, based on said class taps; and 

so learning means for effecting learning so that prediction errors of the prediction values of said teacher data 

obtained on predictive calculations using said class-based tap coefficients and pupil data will be statistically 
minimum to find said tap coefficients from class to dass. 

92. The data processing apparatus according to dalm 91 wherein said learning means effects learning so that a pre- 
ss diction error of the prediction value of sad teacher data obtained on linear predictive calculations using said tap 

coefficients and pupil data will be statistically minimum 

93. The data processing apparatus according to claim 91 wherein said generating means generates said pupil data 
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at least on discrete cosine transforming said data.. 

94. The data processing apparatus according to claim 93 wherein said classification means finds the class of said 
teacher data of interest based on the power of DC or AC components of discrete cosine transformed data which 

5 is said converted data as said class tap. 

95. The data processing apparatus according to claim 91 wherein said class tap extraction means extracts said pupil 
data as said class tap from blocks neighboring to a block associated with teacher data of Interest 

10 96. The data processing apparatus according to claim 91 wherein said class tap extraction means extracts said pupil 
data as said class taps even from the block corresponding to the teacher data of Interest. 

97. The data processing apparatus according to claim 91 wherein said data Is picture data of a moving picture or a 
still picture. 

13 

98. A data processing method for learning, from converted data based on a preset block as a unit, obtained at least 
on orthogonal or frequency transforming data in terms of said preset block as a unit, tap coefficients used for 
finding, by predictive calculations, processed data which Is said converted data processed In a desired fashion, 
comprising the steps of: 

20 

generating said block-based converted data as pupil data as pupil In learning; 

extracting pupil data used for assigning teacher data of interest to one of a plurality of classes, at least from 
a block other than a block corresponding to said teacher data of Interest, with said processed data as teacher 
data as teacher In learning, and for outputtlng the extracted data as class taps; 
25 finding a class of said teacher data of interest, based on said class taps; and 

effecting learning so that prediction errors of the prediction values of said teacher data obtained on predictive 
calculations using said class-based tap coefficients and pupil data will be statistically minimum to find said tap 
coefficients from class to class. 

30 99. A recording medium having recorded thereon a program for having acomputer execute data processing of learning, 
from converted data based on a preset block as a unit, obtained at least on orthogonal or frequency transforming 
data in terms of said preset block as a unit, tap coefficients used for finding, by predictive calculations, processed 
data which is said converted data processed in a desired fashion, said program Including the steps of: 

35 generating said block-based converted data as pupil data as pupil In learning; 

extracting pupil data used for assigning teacher data of interest to one of a plurality of classes, at least from 
a block other than a block corresponding to said teacher data of interest, with said processed data as teacher 
data as teacher In learning, and for outputtlng the extracted data as class taps; 
finding a class of said teacher data of interest, based on said class taps; and 

effecting learning so that prediction errors of the prediction values of said teacher data obtained on predictive 
calculations using said class-based tap coefficients and pupil data will be statistically minimum to find said tap 
coefficients from class to class. 

1 00.A data processing apparatus for processing block-based converted data, obtained on applying at least orthogonal 
45 translorm or frequency transform to data In terms of a preset block as a unit, comprising: 

acquisition means for acquiring tap coefficients as found on learning; 

prediction tap extraction means for extracting said converted data used for predictive calculations for decoding 
data of a data block of interest among data blocks, each of which is a block of said data, at least from a 
so converted block as a block of said converted data, corresponding to a data block other than the data block ot 

Interest, to output the extracted data as prediction taps; and 

calculating means for performing preset predictive calculations, using said tap coefficients and prediction taps, 
to decode the converted data to original data. 

55 101 .The data processing apparatus according to claim 100 wherein said calculating means performs linear predictive 
calculations, using said tap coefficients and the prediction tap, to decode said converted data to original data. 

102.The data processing apparatus according to claim 1 00 further comprising: 
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storage means having said tap coefficients stored therein; 

said acquisition means acquiring saJd tap coefficients from said storage means. 

103 . The data processing apparatus according to claim 100 wherein said converted data is said original data at least 
5 discrete cosine transformed. 

104. The data processing apparatus according to claim 100 further comprising: 

class tap extraction means for extracting said converted data used for assigning data of interest to one of a 
10 plural number of classes from data of said data block of Interest to output the extracted data as class tap; and 

classification means for finding the class of said data of interest, based on said class tap, by way of classifi- 
cation; 

said calculation means performing predictive calculations, using said predictive tap and said tap coefficients 
corresponding to the class of said data of Interest 

13 

1 06 . The data processing apparatus according to claim 1 00 wherein said predictive tap extraction means extracts said 
converted data to be used as said prediction taps from said converted blocks corresponding to data blocks neigh- 
boring to said data block of Interest. 

20 1 06.The data processing apparatus according to claim 1 00 wherein said predictive tap extraction means extracts said 
converted data to be used as said prediction tap from said converted block corresponding to said data block of 
interest and from the converted blocks corresponding to data blocks other than the data block of Interest. 

1 07. The data processing apparatus according to claim 1 00 wherein said predictive tap extraction means extracts said 
25 converted data showing position relationships indicating larger correlation with respect to data of interest among 

data of said data block of interest as said prediction tap. 

1 05. The data processing apparatus according to claim 1 07 wherein said predictive tap extraction means extracts said 
converted data showing position relationships Indicating correlation with respect to data of Interest larger than a 

30 preset threshold value as said prediction tap.. 

109 . The data processing apparatus according to claim 1 07 wherein said predictive tap extraction means extracts said 
converted data showing position relationships Indicating correlation with respect to data of Interest within a preset 
rank value. 

36 

110. The data processing apparatus according to claim 100 wherein said tap coefficients have been acquired by learn- 
ing, using said tap coefficients and the converted data, so that a prediction error of the prediction value of said 
original data obtained on preset predictive calculations will be statistically mlnmum. 

#> in .The data processing apparatus according to claim 1 00 wherein said original data Is picture data of a moving picture 
or a still picture. 

112. A data processing method for processing block-based converted data, obtained on applying at least orthogonal 
transform or frequency transform to data in terms of a preset block as a unit, comprising the steps of 

45 

acquiring tap coefficients as found on learning; 

extracting said converted data used for predictive calculations for decoding data of a data block of interest 
among data blocks each of which is a block of said data, at least from a converted block as a block of said 
converted data, corresponding to a data block other than the data block of interest, to output the extracted 
so data as prediction taps; and 

performing preset predictive calculations, using said tap coefficients and prediction taps, to decode the con- 
verted data to original data. 

113. A recording medium having recorded thereon a program for having a computer execute data processing of process- 
es ing block-based converted data, obtained on applying at least orthogonal transform or frequency transform to data 

in terms of a preset block as a unit said program comprising the steps of: 

acquiring tap coefficients as found on learning; 
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extracting said converted data used for predictive calculations for decoding data of a data block of Interest 
among data blocks each of which Is a block of said data, at least from a converted block as a block of said 
converted data, corresponding to a data block other than the data block of Interest, to output the extracted 
data as prediction taps; and 

5 performing preset predictive calculations, using said tap coefficients and prediction taps, to decode the con- 

verted data to original data. 

1 1 4.A data processing apparatus for learning tap coefficients used for decoding, by predictive calculations, block-based 
converted data obtained at least on applying orthogonal or frequency transform to data In terms of a preset block 
10 as a unit, comprising: 

generating means for generating pupil data as pupil, by at least applying orthogonal or frequency transform 
to teacher data as teacher; 

prediction tap extracting means for extracting said pupil data used for predictive calculations used for decoding 
15 teacher data of a teacher block of Interest among teacher blocks each being a block of said teacher data, from 

at least a block of pupils as block of said pupil data corresponding to the teacher blocks other than the teacher 
block of Interest, to output the extracted data as predictive taps; and 

learning means for effecting learning so that predictive errors of the prediction values of said teacher data 
obtained on performing predictive calculations using the tap coefficients and the prediction taps will be statis- 
20 tlcally minimum to find tap coefficients. 

HS.The data processing apparatus according to claim 114 wherein said learning means effects learning so that a 
prediction error of the prediction value of said teacher data obtained on effecting linear predictive calculations 
using the tap coefficients and the pupil data will be statistically minimum. 

25 

. 116.The data processing apparatus according to claim 114 wherein said generating means at least discrete cosine 
transforms said teacher data to generate said pupil data. 

117.The data processing apparatus according to claim 114 further comprising: 

30 

class tap extraction means for extracting said pupil data used for assigning teacher data of interest among 
said teacher data of said teacher blocks of Interest to one of a plural number of classes to output the extracted 
data as class tap; and 

classification means for finding the class of said teacher data of Interest, based on said class tap, by way ol 
35 classification; 

said learning means effecting learning so that a predictive error ofthe prediction value of said teacher data 
obtained on performing predictive calculations using the prediction taps and the tap coefficients corresponding 
to the class or said teacher data or Interest will be statistically minimum to find tap coefficients. 

40 H8.The data processing apparatus according to claim 114 wherein said prediction tap extraction means extracts said 
pupil data aa said prediction taps from said pupil blocks corresponding to teacher blocks neighboring to said teacher 
block of interest. 

1 1 g.The data processing apparatus according to claim 1 14 wherein said prediction tap extraction means extracts said 
pupil data as said prediction taps from said pupil block corresponding to said teacher block ol Interest and from 
pupil blocks corresponding to teacher blocks other than said teacher block of Interest. 

120.The data processing apparatus according to claim 1 14 wherein said prediction tap extraction means extracts said 
pupil data showing position relationships indicating larger correlation with respect to teacher data of interest among 
50 teacher data of said teacher block of interest 

1 21 .The data processing apparatus according to dalm 1 20 wherein said prediction tap extraction means extracts said, 
pupil data showing position relationships Indicating correlation with respect to teacher data of Interest not lower 
than a preset threshold value. 

55 

122.The data processing apparatus according to claim i20 wherein said prediction tap extraction means extracts said 
pupil data showing position relationships Indicating correlation with respect to teacher data of interest within a 
preset of rank value. 
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123 . The data processing apparatus according to claim 114 wherein said teacher data is data of a moving picture or a 
still picture. 

124. A data processing method for learning tap coefficients used for decoding, by predictive calculations, block-based 
converted data obtained at least on applying orthogonal or frequency transform to data In terms of a preset block 
as a unit, comprising the steps of: 

generating pupil data as pupil, by at least applying orthogonal orf requency transform to teacher data as teach- 
er; 

extracting said pupil data used for predictive calculations used for decoding teacher data of a teacher block 
of Interest among teacher blocks each being a block of said teacher data, from at least a block of pupils as 
block of seJd pupil data corresponding to the teacher blocks other than the teacher block of Interest, to output 
the extracted data as predictive taps; and 

effecting learning so that predictive errors of the prediction values of said teacher data obtained on performing 
predictive calculations using the tap coefficients and the prediction taps will be statistically minimum to find 
tap coefficients. 

1 25. A recording medium having recorded thereon a program Tor having a computer execute data processing of learning 
tap coefficients used for decoding, by predictive calculations, block-based converted data obtained at least on 
applying orthogonal or frequency transform to data In terms of a preset block as a unit, said program comprising 
the steps of: 

generating pupil data as pupil, by at least applyln g orthogonal orf requency transform to teacher data as teach- 



extracting said pupil data used for predictive calculations used for decoding teacher data of a teacher 
block of Interest among teacher blocks each being a block of said teacher data, from at least a block o1 
pupils as block of said pupil data corresponding to the teacher block other than the teacher block of Interest, 
to output the extracted data as predictive taps; and 

effecting learning so that predictive errors of the prediction values of said teacher data obtained on per- 
forming predictive calculations using the tap coefficients and the prediction taps will be statistically mini- 
mum to find said tap coefficients. 

126 . A data processing apparatus lor learning an extraction pattern of first data to be used as prediction taps used for 
predictive calculations when second data is to be acquired by predictive calculations using said prediction taps 
extracted from said first data and preset tap coefficients, comprising: 

correlation calculating means for finding the correlation of said first data showing a preset position relationship 
with respect to second data lying In a preset position, said first and second data being used for learning said 
extraction pattern, from one said position relationship to another; and 

setting means for setting the extraction pattern of said first data to be used as said prediction taps used for 
said predictive calculations executed for obtaining said second data, based on said correlation. 

127. The data processing apparatus according to claim 125 wherein said correlation calculating means finds statistic 
correlation between said first data in a preset position relationship with respect to said second data lying In a preset 
position and said second data lying In the preset position. 

128. The data processing apparatus according to darn 126 wherein said setting means sets the position pattern of 
said first data, showing a position relationship indicating a large value of said correlation, as said extraction pattern. 

129 . The data processing apparatus according to darn 1 26 wherein said setting means sets the position pattern ot 
said first data, showing a position relationship Indicating said correlation not less than a preset threshold value, 
as said extraction pattern. 

130 . The data processing apparatus according to darn 1 26 wherein said setting means sets the position pattern of 
said first data showing a posiUon relationship indicating said correlation within a preset rank value as said extraction 
pattern. 
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131 .A data processing method tor learning an extraction pattern of first data to be used as prediction taps used for 
predictive calculations when second data Is to be acquired by predictive calculations using said prediction taps 
extracted from said first data and preset tap coefficients, comprising the steps of: 



finding the correlation of said first data showing a preset position relationship with respect to second data lying 
In a preset position, said first and second data being used for learning said extraction pattern, from one said 
position relationship to another, and 

setting the extraction pattern of said first data to be used as said prediction taps used for said predictive 
calculations executed for obtaining said second data, based on said correlation. 

132.A recording medium having recorded thereon a program for having a computer execute data processing of learning 
an extraction pattern of first data to be used as prediction taps used tor predictive calculations when second data 
Is to be acquired by predictive calculations using said prediction taps extracted from said first data and preset tap 
coefficients, said program comprising the steps of: 

finding the correlation of said first data showing a preset position relationship with respect to second data lying 
in a preset position, said first and second data being used for learning said extraction pattern, from one said 
position relationship to another; and 

setting the extraction pattern of said first data to be used as said prediction taps used for said predictive 
calculations executed for obtaining said second data, based on said correlation. 
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